Windows PostgreSQL and the case of the vanishing libint.dll

My PostgreSQL 9.0.3 databases running on Windows Server 2008 R2 64-bit were refusing to start. The server was rebooted unexpectedly with no clean shutdown and when it came back up, PostgreSQL didn’t. I always have such fun with Windows servers, so here we go…

1. Open the Services control panel and try to start the postgresql-x64-9.0 service:

Windows could not start the postgresql-x64-9.0 – PostgreSQL Server 9.0 service on Local Computer.

Error 1053: The service did not respond to the start or control request in a timely fashion.

This error pops up immediately; there is no real timeout in play here. There was nothing in the PostgreSQL log file, so it must be something to do with Windows. Some Googling tells me to make sure the user starting the service is not an administrator. It uses the postgres user, and I don’t know how to check if it has administrator privileges as Windows seems to deny its existence when I open the users from the control panel.

The Security Events show a successful logon audit for the postgres user so there’s no password issues.

2. Try to start it manually from a DOS prompt:

C:\Users\ntm\pg_ctl start
The program can't start because libintl.dll is missing from your computer. 

Try reinstalling the program to fix this problem.

I see libintl-8.dll under C:\Program Files (x86)\psqlODBC\0900\bin, but nothing under C:\Program Files\psqlODBC\0900\bin or C:\Program Files\PostgreSQL\9.0\bin, and no libintl.dll file anywhere.

3. I added the Program Files (x86) directory to the path environment variable, copied the file to libintl.dll, and this time got:

The application was unable to start correctly (0xc000007b).

OK, so I assume that file was a 32-bit library, where as I need a 64-bit one.

4. I reinstalled PostgreSQL 9.0.3 64-bit. This failed with the message:

Problem running post-install step. Installation may not complete correctly.
Failed to start the database server.

When I checked, it did put libintl.dll in C:\Program Files\PostgreSQL\9.0\bin which is progress I guess.

5. Let’s try to start the service via Control Panel:

The specified service has been marked for deletion.

I closed and re-opened the services page, and then it had disappeared completely.

6. OK, let’s try starting it manually again:

C:\Users\ntm>runas /user:postgres "pg_ctl start"
Enter the password for postgres:
Attempting to start pg_ctl start as user "BPPWIN01-CL-SQL\postgres" ...

Some screen popped up briefly, disappeared, didn’t work and didn’t tell me what was wrong. Awesome.

7. Reinstalled. Again. This time the installer ran to success (yay) but wants to restart (grrr). I’ll take that under advisement and keep going.

8. The service is back, so tried to start it.

Windows could not start the postgresql-x64-9.0 – PostgreSQL Server 9.0 service on Local Computer.

Error 1053: The service did not respond to the start or control request in a timely fashion.

This time it did take forever before erroring. I checked the security on all the PostgreSQL folders and found that the data directory did not have read/write access for the postgres user.

9. Give postgres user access to its own data, and tried to start the service.

Success! I can now logon. So two questions remain:

1. Why did it lose its dll file/security/whatever else I didn’t see? Was it the unclean shutdown, or some recent patch, or something else just waiting for a reboot to screw me over?
2. Why does Windows hate me so much?