Today's high-bandwith exercise also showed me that I have pgbouncer logs dating back to 2022-09-29
% head -1 /var/log/pgbouncer/pgbouncer.log
2022-09-29 18:46:57.947 UTC [97970] ERROR To allow TLS connections from clients, client_tls_key_file and client_tls_cert_file must be set.
% wc -l /var/log/pgbouncer/pgbouncer.log
5328700 /var/log/pgbouncer/pgbouncer.log
Of those 5.3 million entries, 3.4 million are telling me that I don't have have connection capacity.
% grep -c 'no more connections allowed' /var/log/pgbouncer/pgbouncer.log
3427769
Let's see the busiest days:
% grep 'no more connections allowed' /var/log/pgbouncer/pgbouncer.log > ~/tmp/bad-connections
First day:
% head -1 ~/tmp/bad-connections
2022-09-30 17:51:29.327 UTC [20497] WARNING C-0x8015bbb10: (nodb)/(nouser)@127.163.0.80:65441 pooler error: no more connections allowed (max_client_conn)
Get a count
% cut -f 1 -w ~/tmp/bad-connections | sort | uniq -c > ~/tmp/bad-connections.day.count
301 bad connection days
% wc -l ~/tmp/bad-connections.day.count
301 /usr/home/dvl/tmp/bad-connections.day.count
my top 30 days:
% sort -rn ~/tmp/bad-connections.day.count | head -30
1252069 2026-04-22
327400 2026-04-21
296789 2024-12-01
140700 2026-04-20
137068 2026-04-19
92760 2026-04-02
80634 2025-01-22
78216 2025-09-11
78094 2025-09-10
74338 2025-10-11
60068 2024-10-10
49015 2023-05-08
47293 2024-03-27
44453 2025-09-19
43206 2022-12-15
40506 2024-02-18
39326 2025-01-21
35266 2022-10-10
29787 2024-01-24
27428 2025-10-10
26158 2025-01-20
24899 2024-06-23
24708 2024-10-09
24589 2024-05-15
23888 2024-04-16
23029 2024-05-31
20053 2024-03-21
19197 2026-01-15
18276 2026-04-10
11908 2024-10-11