Anyway, I may or may not write about Oracle here – there’s so much else out there already I don’t feel like I can contribute much else.

$ psql mydb -f postgis.sql
SET
BEGIN
psql:postgis.sql:59: ERROR:  could not load library "/usr/pgsql-9.0/lib/postgis-1.5.so": /usr/pgsql-9.0/lib/postgis-1.5.so: undefined symbol: GEOSHausdorffDistance
psql:postgis.sql:65: ERROR:  current transaction is aborted, commands ignored until end of transaction block
psql:postgis.sql:70: ERROR:  current transaction is aborted, commands ignored until end of transaction block
-snip-

After a lot of head banging (the brick wall kind, not the heavy metal kind), I figured out I still had the old version of the GEOS library:

# yum list geos
Loaded plugins: rhnplugin, security
Excluding Packages from rhel5s-x86_64 - Base (RPData)
Finished
Excluding Packages from rhel5s-x86_64 - Updates (RPData)
Finished
Installed Packages
geos.x86_64                                                                                                    2.2.3-2.rhel5                                                                                                     installed
Available Packages
geos.x86_64                                                                                                    3.2.2-1.rhel5                                                                                                     postgresql9

A quick upgrade of GEOS to v3 and all was well.

$ psql mydb -f postgis.sql
SET
BEGIN
CREATE FUNCTION
CREATE FUNCTION
-snip-

I messed up the auto start service while testing out some upgrades and needed to reinstall the postgresql 8.3 server package:

# rpm -i postgresql-server-8.3.13-1PGDG.x86_64.rpm
package postgresql-server-8.3.13-1PGDG.x86_64 is already installed

Umm OK I’ll remove it first then:

# rpm -e postgresql-server-8.3.13-1PGDG.x86_64.rpm
error: package postgresql-server-8.3.13-1PGDG.x86_64.rpm is not installed

Umm OK. What does yum have to say:

# yum list installed postgresql\*
Loaded plugins: rhnplugin, security

Installed Packages
postgresql.x86_64 8.3.13-1PGDG installed
postgresql-contrib.x86_64 8.3.13-1PGDG installed
postgresql-libs.x86_64 8.3.13-1PGDG installed
postgresql-plperl.x86_64 8.3.13-1PGDG installed
postgresql-plpython.x86_64 8.3.13-1PGDG installed
postgresql-pltcl.x86_64 8.3.13-1PGDG installed
postgresql-server.x86_64 8.3.13-1PGDG installed

So yum thinks it’s installed. Let’s try removing it with yum:

# yum remove postgresql-server
Loaded plugins: rhnplugin, security

============================================================================================================================================================================================================================================
Package Arch Version Repository Size
============================================================================================================================================================================================================================================
Removing:
postgresql-server x86_64 8.3.13-1PGDG installed 12 M
Removing for dependencies:
postgresql-plperl x86_64 8.3.13-1PGDG installed 72 k
postgresql-plpython x86_64 8.3.13-1PGDG installed 49 k
postgresql-pltcl x86_64 8.3.13-1PGDG installed 50 k

Transaction Summary
============================================================================================================================================================================================================================================
Install 0 Package(s)
Update 0 Package(s)
Remove 4 Package(s)

Is this ok [y/N]: y
Downloading Packages:
Running rpm_check_debug
Running Transaction Test
Finished Transaction Test
Transaction Test Succeeded
Running Transaction
error reading information on service postgresql: No such file or directory
Erasing : postgresql-pltcl 1/4
error: %preun(postgresql-server-8.3.13-1PGDG.x86_64) scriptlet failed, exit status 1
Erasing : postgresql-plpython 2/4
Erasing : postgresql-plperl 3/4

Removed:
postgresql-server.x86_64 0:8.3.13-1PGDG

Dependency Removed:
postgresql-plperl.x86_64 0:8.3.13-1PGDG postgresql-plpython.x86_64 0:8.3.13-1PGDG postgresql-pltcl.x86_64 0:8.3.13-1PGDG

Complete!

Well it removed the dependency language packages, but not the server package. Fair enough, the postgresql service is indeed missing but why should that stop the uninstall?

# chkconfig --list postgresql
error reading information on service postgresql: No such file or directory

I added the service back manually by recreating the /etc/init.d/postgresql file (copied a different server) and then made sure it could add and recognise the service:

# chkconfig --add postgresql
# chkconfig --list postgresql
postgresql 0:off 1:off 2:off 3:off 4:off 5:off 6:off

Now a yum remove works:

# yum remove postgresql-server
Loaded plugins: rhnplugin, security

============================================================================================================================================================================================================================================
Package Arch Version Repository Size
============================================================================================================================================================================================================================================
Removing:
postgresql-server x86_64 8.3.13-1PGDG installed 12 M

Transaction Summary
============================================================================================================================================================================================================================================
Install 0 Package(s)
Update 0 Package(s)
Remove 1 Package(s)

Is this ok [y/N]: y
Downloading Packages:
Running rpm_check_debug
Running Transaction Test
Finished Transaction Test
Transaction Test Succeeded
Running Transaction
Erasing : postgresql-server 1/1
warning: /var/lib/pgsql/.bash_profile saved as /var/lib/pgsql/.bash_profile.rpmsave

Removed:
postgresql-server.x86_64 0:8.3.13-1PGDG

Complete!

Yay, now I know what to do the next time I mess it up… in about 5 mins.

$ db_running.sh
   datname   | procpid | usename  |           sql            |       age       | query_ip | locktype |      mode       | hold_sql | blocker_pid | blocker_ip
-------------+---------+----------+--------------------------+-----------------+----------+----------+-----------------+----------+-------------+------------
 mydb        |   10371 | postgres | drop schema old cascade; | 00:05:17.994846 |          | relation | AccessShareLock |          |             |

So something is blocking the drop, but there are no details as to what who it is. Looking at the pg_locks rows of interest:

mydb=# select * from pg_locks where relation=437506;
 locktype | database | relation | page | tuple | virtualxid | transactionid | classid | objid | objsubid | virtualtransaction |  pid  |        mode         | granted
----------+----------+----------+------+-------+------------+---------------+---------+-------+----------+--------------------+-------+---------------------+---------
 relation |    16384 |   437506 |      |       |            |               |         |       |          | 2/5                | 10371 | AccessExclusiveLock | f
 relation |    16384 |   437506 |      |       |            |               |         |       |          | -1/30795916        |       | AccessShareLock     | t
(2 rows)

Whatever is blocking my session has no PID or transaction. From past experience, I know that weird locking behavior like this is usually due to prepared transactions (gaah I hate them!). Sure enough:

postgres=# select * from pg_prepared_xacts ;
 transaction |                                                                    gid                                                                    |           prepared            |  owner   |  database
-------------+-------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------+----------+-------------
    30795916 | 4871251_a5kEAHHYB2tlcHBsaW4wMS1nZmgscnBwLXNpdGVhLW5vZGUwMS1pbnN0MDEsUDMzNzAw_ZXBwbGluMDEtZ2ZoLHJwcC1zaXRlYS1ub2RlMDEtaW5zdDAxLFAzMzcwMCwB | 2011-07-27 23:56:24.371114+10 | postgres | mydb
(1 row)

It’s a few days old and the owning process is dead. Let’s remove it:

postgres=# \c mydb
You are now connected to database "mydb".
mydb=# rollback prepared '4871251_a5kEAHHYB2tlcHBsaW4wMS1nZmgscnBwLXNpdGVhLW5vZGUwMS1pbnN0MDEsUDMzNzAw_ZXBwbGluMDEtZ2ZoLHJwcC1zaXRlYS1ub2RlMDEtaW5zdDAxLFAzMzcwMCwB';
ROLLBACK PREPARED

And my blocked session immediate resumed and completed the drop.

1. Open the Services control panel and try to start the postgresql-x64-9.0 service:

Windows could not start the postgresql-x64-9.0 – PostgreSQL Server 9.0 service on Local Computer.

Error 1053: The service did not respond to the start or control request in a timely fashion.

This error pops up immediately; there is no real timeout in play here. There was nothing in the PostgreSQL log file, so it must be something to do with Windows. Some Googling tells me to make sure the user starting the service is not an administrator. It uses the postgres user, and I don’t know how to check if it has administrator privileges as Windows seems to deny its existence when I open the users from the control panel.

The Security Events show a successful logon audit for the postgres user so there’s no password issues.

2. Try to start it manually from a DOS prompt:

C:\Users\ntm\pg_ctl start
The program can't start because libintl.dll is missing from your computer. 

Try reinstalling the program to fix this problem.

I see libintl-8.dll under C:\Program Files (x86)\psqlODBC\0900\bin, but nothing under C:\Program Files\psqlODBC\0900\bin or C:\Program Files\PostgreSQL\9.0\bin, and no libintl.dll file anywhere.

3. I added the Program Files (x86) directory to the path environment variable, copied the file to libintl.dll, and this time got:

The application was unable to start correctly (0xc000007b).

OK, so I assume that file was a 32-bit library, where as I need a 64-bit one.

4. I reinstalled PostgreSQL 9.0.3 64-bit. This failed with the message:

Problem running post-install step. Installation may not complete correctly.
Failed to start the database server.

When I checked, it did put libintl.dll in C:\Program Files\PostgreSQL\9.0\bin which is progress I guess.

5. Let’s try to start the service via Control Panel:

The specified service has been marked for deletion.

I closed and re-opened the services page, and then it had disappeared completely.

6. OK, let’s try starting it manually again:

C:\Users\ntm>runas /user:postgres "pg_ctl start"
Enter the password for postgres:
Attempting to start pg_ctl start as user "BPPWIN01-CL-SQL\postgres" ...

Some screen popped up briefly, disappeared, didn’t work and didn’t tell me what was wrong. Awesome.

7. Reinstalled. Again. This time the installer ran to success (yay) but wants to restart (grrr). I’ll take that under advisement and keep going.

8. The service is back, so tried to start it.

Windows could not start the postgresql-x64-9.0 – PostgreSQL Server 9.0 service on Local Computer.

Error 1053: The service did not respond to the start or control request in a timely fashion.

This time it did take forever before erroring. I checked the security on all the PostgreSQL folders and found that the data directory did not have read/write access for the postgres user.

9. Give postgres user access to its own data, and tried to start the service.

Success! I can now logon. So two questions remain:

1. Why did it lose its dll file/security/whatever else I didn’t see? Was it the unclean shutdown, or some recent patch, or something else just waiting for a reboot to screw me over?
2. Why does Windows hate me so much?

$ db_running.sh
 datname | procpid | usename  |                                                 sql                                                  |       age        |  query_ip   | L? | locktype | hold_sql | blocker_ip
---------+---------+----------+------------------------------------------------------------------------------------------------------+------------------+-------------+----+----------+----------+------------
 ntm     |   20163 | postgres | <IDLE> in transaction                                                                                | 00:00:00.455019  | 172.17.0.51 | f  |          |          |
 ntm     |   20535 | postgres | <IDLE> in transaction                                                                                | 00:00:00.004358  | 172.17.0.23 | f  |          |          |
 ntm     |   19974 | postgres | COMMIT                                                                                               | -00:00:00.001497 | 172.17.0.53 | f  |          |          |
 ntm     |   20116 | postgres | select contacts0_.customer_poid as customer9_1_, contacts0_.contact_poid as contact1_1_, contacts0_. | -00:00:00.0051   | 172.17.0.21 | f  |          |          |
(4 rows)

I am calculating the age via:

select … , current_timestamp-s.query_start as age, …

The current_timestamp function I’m using will get the time at the start of the current transaction (i.e. my select statement) so that it does not change for each row being returned (doco reference). This is pretty much always want you want.

However when querying the pg_stat_activity view, it does not actually return the session details as at the start of your transaction but rather it returns the session details as at the time you first query it inside your transaction. Or in the words of the documentation:

“information about the current queries of all sessions is collected when any such information is first requested within a transaction”

What I guess is happening is that my select query starts, gets the current_timestamp value, then a few milliseconds later calls the pg_stat_activity view which can potentially see sessions that started just after my original select started. Hence current_timestamp – query_start becomes negative!

A quick scan of the available date/time functions revealed clock_timestamp() which always returns the timestamp at the time the function is called and is not transaction aware. This sounds like a good alternative for my script, and testing it out confirms my theory:

$ db_running.sh
 datname | procpid | usename  |                                                 sql                                                  |       age        |      age2       | client_addr | L? | locktype | hold_sql | client_addr
---------+---------+----------+------------------------------------------------------------------------------------------------------+------------------+-----------------+-------------+----+----------+----------+-------------
 ntm     |   14874 | postgres | <IDLE> in transaction                                                                                | 00:00:09.15277   | 00:00:09.161446 | 172.17.0.52 | f  |          |          |
 ntm     |   14770 | postgres | COMMIT PREPARED '4871251_B/sBAG7s7uFhcHBsaW4wMy1nbGFzcyxycHAtc2l0ZWItbm9kZTAzLWluc3QwMixQMzM3MDE=_YX | 00:00:00.027994  | 00:00:00.036554 | 172.17.0.53 | f  |          |          |
 ntm     |   14873 | postgres | COMMIT                                                                                               | 00:00:00.009249  | 00:00:00.017878 | 172.17.0.52 | f  |          |          |
 ntm     |   13361 | postgres | <IDLE> in transaction                                                                                | -00:00:00.004689 | 00:00:00.004038 | 172.17.0.21 | f  |          |          |
(4 rows)

I’ve yet to see a negative value re-appear after using this new function. My OCD can breathe a little easier now

You can download my db_running.sh script here.

LOG:  process 916 still waiting for ShareLock on transaction 378662803 after 1000.762 ms
STATEMENT:  update user_customers set last_changed=$1 where id=$20

And a bit later

LOG:  process 916 acquired ShareLock on transaction 378662803 after 1777.653 ms
STATEMENT:  update user_customers set last_changed=$1 where id=$20
LOG:  duration: 2030.106 ms  execute : update user_customers set last_changed=$1 where id=$20

The problem went away before I could run a query to find out what process was holding the lock, and I’ve never been able to catch it in the act. What can we tell from this logged information?

1. The type of lock that it was trying to grab was a Share lock.
2. The statement that is trying to grab a lock is an UPDATE.
3. The object being locked is a Transaction (not tuple).
4. There was only the one duration line, so whatever is holding the lock was quick (<1 sec).

A Share lock would mean the locking process is holding one of the following on the transaction:
• Row exclusive (update, delete, insert)
• Share update exclusive (vacuum, analyze)
• Share row exclusive (not used automatically by postgres processes)
• Exclusive (not used automatically by postgres processes)
• Access exclusive (alter, drop, truncate, reindex, cluster, vacuum full)

I know this particular database has no DDL running and the code does not perform any explicit locking so I can rule out the last 3, leaving Row Exclusive and Share Update Exclusive. I ruled out any autovacuum processing from blocking the updates in my previous post.

So that just leaves Row Exclusive as the possible cause of this blocking. However, remember that my lock is waiting on a transaction not a row. This indicates that some session has a Row Exclusive lock and an open transaction that my session needs to wait on. The only possible explanation (that I can come up with) is that this other session is locking the same row I am trying to update – which doesn’t make a whole lot of sense as the row is very user specific and multiple sessions updating a single user shouldn’t happen in this application.

The other confusing part is that this lock is being held for long enough to put a warning in the log file (>1 sec) and this update should run pretty much instantaneously. Also, the locking session ran fast as it didn’t display any DURATION line which is also set to a 1 sec threshold.

Enough guessing – I took my findings to the developers and together we eventually discovered a scheduled process that batch up requests to update these user rows, and it was possible for duplicate requests of the same rows to be run across multiple parallel sessions. The way it works indicated that many updates could be performed before it would commit, which can explain why my blocked update had to wait so long.

Timeline:

So only session 2 writes stuff to the log which is exactly what I was seeing.

Session 1 – run a VACUUM on a large table (so it takes a while)

mydb=# vacuum big_table;

Session 2 – Update some rows in the table

mydb=# begin;
BEGIN
mydb=# update big_table set cola = 'x' where id = 123456789;
UPDATE 112

Session 3 – Examine pg_locks

This is the VACUUM session:

mydb=# select * from pg_locks where pid=2136;
  locktype  | database | relation | page | tuple | virtualxid | transactionid | classid | objid | objsubid | virtualtransaction | pid  |           mode           | granted
------------+----------+----------+------+-------+------------+---------------+---------+-------+----------+--------------------+------+--------------------------+---------
 relation   | 25693263 | 27681060 |      |       |            |               |         |       |          | 118/92993          | 2136 | RowExclusiveLock         | t
 relation   | 25693263 | 27681056 |      |       |            |               |         |       |          | 118/92993          | 2136 | RowExclusiveLock         | t
 relation   | 25693263 | 27679696 |      |       |            |               |         |       |          | 118/92993          | 2136 | ShareUpdateExclusiveLock | t
 virtualxid |          |          |      |       | 118/92993  |               |         |       |          | 118/92993          | 2136 | ExclusiveLock            | t
 relation   | 25693263 | 27681058 |      |       |            |               |         |       |          | 118/92993          | 2136 | RowExclusiveLock         | t
 relation   | 25693263 | 27681059 |      |       |            |               |         |       |          | 118/92993          | 2136 | RowExclusiveLock         | t
(6 rows)

So the VACUUM has taken a ShareUpdateExclusiveLock on the table and a number of RowExclusiveLock – one for each index.

This is the UPDATE session:

mydb=# select * from pg_locks where pid=20666;
   locktype    | database | relation | page | tuple | virtualxid | transactionid | classid | objid | objsubid | virtualtransaction |  pid  |       mode       | granted
---------------+----------+----------+------+-------+------------+---------------+---------+-------+----------+--------------------+-------+------------------+---------
 relation      | 25693263 | 27681056 |      |       |            |               |         |       |          | 98/115834          | 20666 | RowExclusiveLock | t
 relation      | 25693263 | 27679696 |      |       |            |               |         |       |          | 98/115834          | 20666 | RowExclusiveLock | t
 virtualxid    |          |          |      |       | 98/115834  |               |         |       |          | 98/115834          | 20666 | ExclusiveLock    | t
 relation      | 25693263 | 27681060 |      |       |            |               |         |       |          | 98/115834          | 20666 | RowExclusiveLock | t
 transactionid |          |          |      |       |            |     219901739 |         |       |          | 98/115834          | 20666 | ExclusiveLock    | t
 relation      | 25693263 | 27681059 |      |       |            |               |         |       |          | 98/115834          | 20666 | RowExclusiveLock | t
 relation      | 25693263 | 27681058 |      |       |            |               |         |       |          | 98/115834          | 20666 | RowExclusiveLock | t
(7 rows)

The UPDATE has an Exclusive Lock on its virtual transaction id (which every session does) and since it actually updated some rows, it also has an Exclusive Lock on a real transaction id. It also took Exclusive Row locks on the table and three indexes. Note that none of the locks are waiting, so there are no blocking issues here.

In other words – a VACUUM does not block an UPDATE. This is actually what I expected from looking at this table in the documentation:

SELECT id
FROM property_vw
WHERE id IN (list of approx 32K numbers)
AND baths in ('2')
AND beds in ('2', '3')
AND property_type = 'UNIT'
ORDER BY a, few, columns;

The first thing I noticed was the huge IN list of over 32,000 Ids. Frankly, I’m amazed that PostgreSQL even accepts such a query and doesn’t spit out an error message immediately.

The explain analyze for this query is:
(anonymised and colorised via Depesz’s awesome online explain tool)

Sort  (cost=655585.950..655585.950 rows=1 width=81) (actual time=165765.458..165765.629 rows=2432 loops=1)
    Sort Key: quebec_three.romeo_seven, quebec_three.mike_five, quebec_three.yankee_india, quebec_three.quebec_five, quebec_three.six_bravo, quebec_three.romeo_india, quebec_three.five, (lima_sierra(quebec_three.six_mike, 0)), (lima_sierra(quebec_three.charlie_hotel, ''alpha''::six_delta mike_seven))
    Sort Method:  quicksort  Memory: 439kB
  ->  Nested Loop  (cost=0.000..655585.940 rows=1 width=81) (actual time=22.486..165732.870 rows=2432 loops=1)
        ->  Nested Loop Left Join  (cost=0.000..655576.530 rows=1 width=32) (actual time=22.473..165713.367 rows=2432 loops=1)
                Join Filter: (lima_four.three_alpha = kilo.three_alpha)
              ->  Nested Loop  (cost=0.000..655568.090 rows=1 width=36) (actual time=22.461..165692.927 rows=2432 loops=1)
                    ->  Nested Loop  (cost=0.000..655566.960 rows=2 width=40) (actual time=22.451..165680.208 rows=2496 loops=1)
                          ->  Seq Scan on zulu_juliet zulu_kilo  (cost=0.000..627393.710 rows=567 width=4) (actual time=0.021..5377.204 rows=642599 loops=1)
                                  Filter: ((lima_sierra(((xray_xray -> 'six_kilo'::golf))::charlie_foxtrot, 0) = ANY ('oscar_victor'::charlie_foxtrot[])) AND (lima_sierra(((xray_xray -> 'tango'::golf))::charlie_foxtrot, 0) = 2))
                          ->  Index Scan using xray_five on six_alpha kilo  (cost=0.000..49.680 rows=1 width=36) (actual time=0.249..0.249 rows=0 loops=642599)
                                  Index Cond: (kilo.three_alpha = zulu_kilo.three_alpha)
                                  Filter: (kilo.three_alpha = ANY ('three_india'::charlie_foxtrot[]))
                    ->  Index Scan using india on oscar_seven xray_sierra  (cost=0.000..0.550 rows=1 width=4) (actual time=0.004..0.004 rows=1 loops=2496)
                            Index Cond: (xray_sierra.oscar_juliet = kilo.oscar_juliet)
                            Filter: (xray_sierra.victor_echo = 'zulu_echo'::golf)
              ->  Index Scan using four on victor_yankee lima_four  (cost=0.000..8.430 rows=1 width=8) (actual time=0.007..0.007 rows=1 loops=2432)
                      Index Cond: (lima_four.seven = kilo.three_sierra)
        ->  Index Scan using whiskey on yankee_zulu quebec_three  (cost=0.000..9.390 rows=1 width=57) (actual time=0.006..0.006 rows=1 loops=2432)
                Index Cond: (quebec_three.three_alpha = kilo.three_alpha)

It’s driving the query off an initial sequential scan of a large table. It’s estimating the scan will return 567 rows when in fact it returned 642599 rows. I can understand its limited ability to interpret the query when it has such a monster IN list. What I need is for the query to drive off an index lookup on a different table which is more selective.

Now I know what I want, but how to convince PostgreSQL that it’s the right choice?

So in Oracle, I would have tried adding an ORDERED hint (or more likely, a LEADING hint but then I wouldn’t have anything to compare with here!). However PostgreSQL does not support any hints, and most likely never will. What I did find in the PostgreSQL documentation is the join_collapse_limit query planner parameter:

“Setting it to 1 prevents any reordering of explicit JOINs. Thus, the explicit join order specified in the query will be the actual order in which the relations are joined.”

More info available in the documentation.

So join_collapse_limit seems to be the closest thing in PostgreSQL to the Oracle ordered hint. After setting this for my session:

set join_collapse_limit = 1;

I get the following new explain plan:
(anonymised and colorised via Depesz’s awesome online explain tool)

Sort  (cost=1526179.540..1526179.550 rows=1 width=81) (actual time=281.272..281.445 rows=2432 loops=1)
    Sort Key: quebec_three.romeo_seven, quebec_three.mike_five, quebec_three.yankee_india, quebec_three.quebec_five, quebec_three.six_bravo, quebec_three.romeo_india, quebec_three.five, (lima_sierra(quebec_three.six_mike, 0)), (lima_sierra(quebec_three.charlie_hotel, ''alpha''::six_delta mike_seven))
    Sort Method:  quicksort  Memory: 439kB
  ->  Nested Loop Left Join  (cost=90351.160..1526179.530 rows=1 width=81) (actual time=50.434..251.197 rows=2432 loops=1)
          Join Filter: (lima_four.three_alpha = kilo.three_alpha)
        ->  Nested Loop  (cost=90351.160..1526169.630 rows=1 width=85) (actual time=50.423..240.437 rows=2432 loops=1)
              ->  Nested Loop  (cost=90351.160..1526160.230 rows=1 width=36) (actual time=50.413..232.088 rows=2432 loops=1)
                    ->  Nested Loop  (cost=90351.160..1499651.650 rows=2878 width=32) (actual time=50.158..144.098 rows=23920 loops=1)
                          ->  Bitmap Heap Scan on six_alpha kilo  (cost=90351.160..1481550.430 rows=32209 width=36) (actual time=50.145..59.728 rows=32255 loops=1)
                                  Recheck Cond: (three_alpha = ANY ('victor_bravo'::charlie_foxtrot[]))
                                ->  Bitmap Index Scan on xray_five  (cost=0.000..90343.100 rows=32209 width=0) (actual time=49.636..49.636 rows=32255 loops=1)
                                        Index Cond: (three_alpha = ANY ('victor_sierra'::charlie_foxtrot[]))
                          ->  Index Scan using india on oscar_seven xray_sierra  (cost=0.000..0.550 rows=1 width=4) (actual time=0.002..0.002 rows=1 loops=32255)
                                  Index Cond: (xray_sierra.oscar_juliet = kilo.oscar_juliet)
                                  Filter: (xray_sierra.victor_echo = 'zulu_echo'::golf)
                    ->  Index Scan using papa on zulu_juliet zulu_kilo  (cost=0.000..9.200 rows=1 width=4) (actual time=0.003..0.003 rows=0 loops=23920)
                            Index Cond: (zulu_kilo.three_alpha = kilo.three_alpha)
                            Filter: ((lima_sierra(((zulu_kilo.xray_xray -> 'six_kilo'::golf))::charlie_foxtrot, 0) = ANY ('oscar_victor'::charlie_foxtrot[])) AND (lima_sierra(((zulu_kilo.xray_xray -> 'tango'::golf))::charlie_foxtrot, 0) = 2))
              ->  Index Scan using whiskey on yankee_zulu quebec_three  (cost=0.000..9.390 rows=1 width=57) (actual time=0.003..0.003 rows=1 loops=2432)
                      Index Cond: (quebec_three.three_alpha = kilo.three_alpha)
        ->  Index Scan using four on victor_yankee lima_four  (cost=0.000..9.890 rows=1 width=8) (actual time=0.003..0.004 rows=1 loops=2432)
                Index Cond: (lima_four.seven = kilo.three_sierra)

Which ran in only 281ms. Considering all the single page index lookups, I think this is quite fast.

I’m not happy with this solution though, as the view that is being queried has several other views inside it. If any of them were modified to join tables in a different order, the query would stop working again. Manually rewriting the views to make the query reference all tables would reduce the risk of this query breaking again, but of course you lose the benefit of having the views in the first place which is to encapsulate some of the complex logic from any calling queries.

mydb=# DROP SCHEMA foobar;
NOTICE:  view foobar.vendors depends on schema foobar
NOTICE:  rule _RETURN on view foobar.vendors depends on view foobar.vendors
NOTICE:  table foobar.transfer_types depends on schema foobar
NOTICE:  view foobar.suggest depends on schema foobar
NOTICE:  sequence foobar.auction_results_auction_results_poid_seq depends on schema foobar
NOTICE:  default for table foobar.auction_results column auction_results_poid depends on sequence foobar.auction_results_auction_results_poid_seq
NOTICE:  function foobar.array_median(numeric[]) depends on schema foobar
<snip>
ERROR:  cannot drop schema foobar because other objects depend on it
HINT:  Use DROP ... CASCADE to drop the dependent objects too.

Looking closely, I notice that every single problem it highlighted was a dependency of something in that schema with something else within that same schema. So… if I’m dropping the entire schema why are you complaining about it?

Of course it’s easily fixed by using cascade, but I don’t like stupid error messages.

mydb=# drop SCHEMA foobar cascade;
NOTICE:  Blah blah lots of whiney crap here blah blah
DROP SCHEMA