Scaling Postgres

In this episode of Scaling Postgres, we discuss how to use foreign data wrappers, compression available in timescaledb, working with fuzzy search and describing Postgres query execution. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://blog.crunchydata.com/blog/understanding-postgres_fdw https://blog.rustprooflabs.com/2021/08/timescale-compression-openstreetmap-tags https://arctype.com/blog/postgres-fuzzy-search/ https://dev.to/yugabyte/postgres-query-execution-simple-query-2i71 https://www.highgo.ca/2021/08/20/how-batch-insertion-was-done-in-postgres_fdw/ https://dev.to/yugabyte/pgbench-client-jobs-68g https://blog.crunchydata.com/blog/waiting-for-postgis-3.2-secure-cloud-raster-access https://www.migops.com/blog/2021/08/20/validate-postgresql-extension-upgrade-scripts-using-pg_validate_extupgrade/ https://www.cybertec-postgresql.com/en/pg_timetable_v4-is-out/ https://postgresql.life/post/andrey_borodin/ https://www.rubber

In this episode of Scaling Postgres, we discuss new releases for Postgres & pgbouncer, have a performance discussion, describe the different index types and how to use them. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.postgresql.org/about/news/postgresql-134-128-1113-1018-9623-and-14-beta-3-released-2277/ https://www.pgbouncer.org/2021/08/pgbouncer-1-16-0 https://www.rubberduckdevshow.com/episodes/8-database-performance/ https://pganalyze.com/blog/postgres-create-index https://www.youtube.com/watch?v=gBGUuMrO6MI https://www.cybertec-postgresql.com/en/postgresql-vs-redis-vs-memcached-performance/ https://www.percona.com/blog/impact-of-network-and-cursor-on-query-performance-of-postgresql/ https://www.ongres.com/blog/noset-postgresql-extension-for-blocking-set-reset-execution/ https://www.percona.com/blog/migrating-postgresql-to-kubernetes https://www.highgo.ca/2021/08/09/horizontal-scalability-options-in-postgresq

In this episode of Scaling Postgres, we discuss ways to work around the planner, working with grouping sets, using a lateral join and not-so-easy Postgres issues. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://heap.io/blog/when-the-postgres-planner-is-not-very-smart https://www.cybertec-postgresql.com/en/postgresql-grouping-sets-rollup-cube/ https://fluca1978.github.io/2021/08/07/PostgreSQLLateralJoin.html https://pgdash.io/blog/postgresql-six-not-so-easy-pieces.html https://www.postgresql.fastware.com/blog/logical-decoding-of-two-phase-commits https://blog.crunchydata.com/blog/using-cert-manager-to-deploy-tls-for-postgres-on-kubernetes https://postgresql.life/post/bryn_llewellyn/ https://www.rubberduckdevshow.com/episodes/7-writing-secure_code/

In this episode of Scaling Postgres, we discuss JSONB vs. window functions, a Postgres security checklist, implementing full-text search and using NFL timeseries data. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://swarm64.com/post/postgresql-jsonb-vs-window-functions/ https://www.cybertec-postgresql.com/en/postgresql-security-things-to-avoid-in-real-life/ https://blog.crunchydata.com/blog/postgres-full-text-search-a-search-engine-in-a-database https://blog.timescale.com/blog/hacking-nfl-data-with-postgresql-timescaledb-and-sql/ https://ardentperf.com/2021/07/26/postgresql-logical-replicas-and-snapshots-proceed-carefully/ https://www.pgmustard.com/blog/max-parallel-workers-per-gather https://fluca1978.github.io/2021/07/27/pgbackrestAsync.html https://b-peng.blogspot.com/2021/07/deploying-pgpool2-exporter-with-docker.html https://www.percona.com/blog/improve-postgresql-query-performance-insights-with-pg_stat_monitor/ https:/

In this episode of Scaling Postgres, we discuss the current status of zheap, the gaps in sequences, how to efficiently paginate and how to set up patroni with etcd and haproxy. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.cybertec-postgresql.com/en/postgresql-zheap-current-status/ https://incident.io/blog/one-two-skip-a-few https://pganalyze.com/blog/pagination-django-postgres https://arctype.com/blog/postgres-patroni/ https://fluca1978.github.io/2021/07/20/PostgreSQLExtensions.html https://www.highgo.ca/2021/07/23/the-amazing-buffer-tag-in-postgresql/ https://www.paulox.net/2021/07/19/maps-with-django-part-2-geodjango-postgis-and-leaflet/ https://blog.crunchydata.com/blog/using-postgis-and-pg_featureserv-with-qgis https://blog.crunchydata.com/blog/waiting-for-postgis-3.2-st_makevalid https://blog.crunchydata.com/blog/waiting-for-postgis-3.2-st_contour-and-st_setz https://postgresql.life/post/rafia_sabih/ https:/

In this episode of Scaling Postgres, we discuss understanding lateral joins, generating JSON with Postgres, the dbcritic utility, and fetch first rows vs. limit. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.cybertec-postgresql.com/en/understanding-lateral-joins-in-postgresql/ https://blog.crunchydata.com/blog/generating-json-directly-from-postgres https://www.channable.com/tech/dbcritic-constructively-criticizing-your-postgres-schema https://www.cybertec-postgresql.com/en/postgresql-limit-vs-fetch-first-rows-with-ties/ https://elephanttamer.net/?p=58 https://www.youtube.com/watch?v=znPL6iDDou8 https://fluca1978.github.io/2021/07/13/PostgreSQLWalTraffic.html https://www.postgresql.fastware.com/blog/improved-logging-by-libpq-in-postgresql-14 https://www.highgo.ca/2021/07/14/first-contact-with-the-pg_filedump/ http://amitkapila16.blogspot.com/2021/07/logical-replication-of-in-progress.html https://postgresql.life/post

In this episode of Scaling Postgres, we discuss how to delay replication, working with materialized view permissions, paranoid configuration options and addressing memory overcommit. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.cybertec-postgresql.com/en/simulating-temporal-tables-with-dblink-and-replication-delay/ https://blog.rustprooflabs.com/2021/07/postgres-permission-mat-view https://ardentperf.com/2021/07/06/paranoid-sql-execution-on-postgresql/ https://sosna.de/posts/pgaas-memory-overcommit/ https://pganalyze.com/blog/postgresql-partitioning-django https://www.migops.com/blog/2021/07/02/credcheck-extension-to-enforce-username-and-password-checks-in-postgresql/ https://blog.crunchydata.com/blog/the-next-generation-of-kubernetes-native-postgres https://blog.crunchydata.com/blog/getting-started-with-pgo-postgres-operator-5.0 https://b-peng.blogspot.com/2021/07/postgres-disaster-recovery-on-k8s-zalando.html http

In this episode of Scaling Postgres, we discuss tracking per-operation explain analyze, Postgres 14 contribution stats, parallel execution on foreign servers and incremental view maintenance. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.pgmustard.com/blog/calculating-per-operation-times-in-postgres-explain-analyze https://www.highgo.ca/2021/06/28/some-interesting-statistics-about-pg-14-contributions/ https://www.highgo.ca/2021/06/28/parallel-execution-of-postgres_fdw-scans-in-pg-14-important-step-forward-for-horizontal-scaling/ https://yugonagata-pgsql.blogspot.com/2021/06/implementing-incremental-view.html https://lesovsky.medium.com/updates-of-postgresql-observability-diagram-66f0bbbfe021 https://www.migops.com/blog/2021/07/01/ora2pg-now-supports-oracle_fdw-to-increase-the-data-migration-speed/ http://postgis.net/2021/07/02/postgis-3.1.3/ https://www.cybertec-postgresql.com/en/postgis-setup-with-ubuntu-20-04-2/ https

In this episode of Scaling Postgres, we discuss practical partitioning, understanding pg_repack, explaining buffers and enforcing multi-row constraints. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.postgresql.org/about/news/postgresql-14-beta-2-released-2249/ https://www.postgresvision.com/ https://vyruss.org/blog/practical-partitioning-in-production-with-postgres-postgres-vision-2021.html https://www.percona.com/blog/2021/06/24/understanding-pg_repack-what-can-go-wrong-and-how-to-avoid-it/ https://www.depesz.com/2021/06/20/explaining-the-unexplainable-part-6-buffers/ https://www.cybertec-postgresql.com/en/postgresql-constraints-over-multiple-rows/ https://www.cybertec-postgresql.com/en/from-md5-to-scram-sha-256-in-postgresql/ https://www.depesz.com/2021/06/23/a-tale-of-making-company-wide-standard-psqlrc/ https://fabianlindfors.se/blog/schema-migrations-in-postgres/ https://pganalyze.com/blog/geodjango-postgis htt

In this episode of Scaling Postgres, we discuss Redis vs. Postgres, working in the hard quadrant, how Postgres optimizes joins and working with materialized views. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://spin.atomicobject.com/2021/02/04/redis-postgresql/ https://momjian.us/main/blogs/pgblog/2021.html#June_16_2021 https://www.youtube.com/watch?v=XA3SBgcZwtE https://www.cybertec-postgresql.com/en/creating-and-refreshing-materialized-views-in-postgresql/ https://blog.crunchydata.com/blog/fun-with-pg_checksums https://www.cybertec-postgresql.com/en/cross-join-in-postgresql/ https://paquier.xyz/postgresql-2/postgres-14-monitoring-copy/ https://www.youtube.com/c/percona/videos https://pgsqlpgpool.blogspot.com/2021/06/promoting-specied-node-in-pgpool-ii.html https://blog.timescale.com/blog/how-and-why-to-become-a-postgresql-contributor/ https://postgresql.life/post/josh_berkus/ https://www.rubberduckdevshow.com/

In this episode of Scaling Postgres, we discuss only using bigint, high availability, Postgres autocommit and synchronized sequential scans. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://blog.rustprooflabs.com/2021/06/postgres-bigint-by-default https://www.percona.com/blog/2021/06/11/postgresql-ha-with-patroni-your-turn-to-test-failure-scenarios/ https://www.cybertec-postgresql.com/en/disabling-autocommit-in-postgresql-can-damage-your-health/ https://www.cybertec-postgresql.com/en/data-warehousing-making-use-of-synchronized-seq-scans/ https://blog.crunchydata.com/blog/better-range-types-in-postgres-14-turning-100-lines-of-sql-into-3 https://www.youtube.com/watch?v=gDFIc2uaMc8 https://tapoueh.org/blog/2021/06/postgresql-as-a-microservice/ https://patternmatchers.wordpress.com/2021/06/11/ignore-nulls-in-postgres/ https://paquier.xyz/postgresql-2/postgres-14-memory-dumps/ https://blog.crunchydata.com/blog/postgresql-on-linux

In this episode of Scaling Postgres, we discuss managing autovacuum, better JSON in Postgres 14, how to avoid redundant updates and operating system tuning. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.cybertec-postgresql.com/en/enabling-and-disabling-autovacuum-in-postgresql/ https://blog.crunchydata.com/blog/better-json-in-postgres-with-postgresql-14 https://fluca1978.github.io/2021/06/03/PostgreSQLUpdateTrigger.html https://www.enterprisedb.com/blog/tuning-debian-ubuntu-postgresql https://arctype.com/blog/postgres-performance-monitoring/ https://www.percona.com/blog/2021/06/02/postgres_fdw-enhancement-in-postgresql-14/ https://paquier.xyz/postgresql-2/postgres-14-reindex-tbspace/ https://pgsqlpgpool.blogspot.com/2021/05/dropping-support-for-version-2-protocol.html https://yugonagata-pgsql.blogspot.com/2021/05/implementing-incremental-view.html https://www.highgo.ca/2021/05/31/getting-started-with-pg_bulkload/

In this episode of Scaling Postgres, we discuss how to interpret explain analyze, the best ways to load data in bulk, how to limit rows affected by update or delete and using pg_buffercache. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.cybertec-postgresql.com/en/how-to-interpret-postgresql-explain-analyze-output/ https://www.cybertec-postgresql.com/en/postgresql-bulk-loading-huge-amounts-of-data/ https://blog.crunchydata.com/blog/simulating-update-or-delete-with-limit-in-postgres-ctes-to-the-rescue https://fluca1978.github.io/2021/05/28/PostgreSQLMemoryFuntions.html https://www.percona.com/blog/2021/05/27/new-features-in-postgresql-14-bulk-inserts-for-foreign-data-wrappers/ https://www.postgresql.fastware.com/blog/enhancement-to-target-session-attrs https://www.ongres.com/blog/tracking-materialized-views-refresh-performance-with-mvstat-in-postgresql/ https://seiler.us/2021-05-23-speeding-up-pgbackrest-recovery/ https:

In this episode of Scaling Postgres, we discuss the release of Postgres 14 Beta 1, different PG 14 improvements, choosing the best type of primary key and using composite keys. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.postgresql.org/about/news/postgresql-14-beta-1-released-2213/ https://pganalyze.com/blog/postgres-14-performance-monitoring https://www.cybertec-postgresql.com/en/uuid-serial-or-identity-columns-for-postgresql-auto-generated-primary-keys/ https://www.endpoint.com/blog/2021/05/20/database-design-using-composite-keys https://www.youtube.com/playlist?list=PLownlFUq_rL6txMapX6UUuquqGdUINmkd https://akorotkov.github.io/blog/2021/05/22/pg-14-query-parsing/ https://blog.crunchydata.com/blog/waiting-for-postgis-3.2-st_interpolateraster https://www.migops.com/blog/2021/05/19/porting-array-length-from-oracle-plsql-to-postgresql-plpgsql/ https://blog.crunchydata.com/blog/pgo-4.7-the-crunchy-postgres-operator-now

In this episode of Scaling Postgres, we discuss the newest Postgres releases, implementing a data warehouse, using exclusion operators and setting up read-only users. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.postgresql.org/about/news/postgresql-133-127-1112-1017-and-9622-released-2210/ https://www.narrator.ai/blog/using-postgresql-as-a-data-warehouse/ https://www.youtube.com/watch?v=M58MR1vqnAk https://blog.crunchydata.com/blog/creating-a-read-only-postgres-user https://paquier.xyz/postgresql-2/postgres-14-table-compression/ https://momjian.us/main/blogs/pgblog/2021.html#May_10_2021 https://www.cybertec-postgresql.com/en/setting-up-postgresql-streaming-replication/ https://blog.crunchydata.com/blog/tls-postgres-kubernetes-openssl https://www.enterprisedb.com/blog/regression-analysis-postgresql-tensorflow-part-3-data-analysis https://postgresql.life/post/laurenz_albe/

In this episode of Scaling Postgres, we discuss selecting random rows quickly, faster distinct queries, how replication handles temp and unlogged tables and how to terminate connections. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://blog.hagander.net/getting-random-rows-faster-very-much-faster-249/ https://blog.timescale.com/blog/how-we-made-distinct-queries-up-to-8000x-faster-on-postgresql/ https://fluca1978.github.io/2021/05/10/PostgreSQLUnloggedTablesReplication.html https://www.cybertec-postgresql.com/en/terminating-database-connections-in-postgresql/ https://www.tangramvision.com/blog/creating-postgresql-test-data-with-sql-pl-pgsql-and-python https://momjian.us/main/blogs/pgblog/2021.html#May_3_2021 https://akorotkov.github.io/blog/2021/05/06/jsonpath-double-asterisk-lax/ https://blog.crunchydata.com/blog/using-kubernetes-chances-are-you-need-a-database https://blog.crunchydata.com/blog/the-many-spatial-indexes-of-pos

In this episode of Scaling Postgres, we discuss doing data analysis with SQL, sophisticated pgBouncer set ups, understanding how to avoid deadlocks and an intro to tuning and optimization. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://hakibenita.com/sql-for-data-analysis https://www.enterprisedb.com/postgres-tutorials/pgbouncer-setup-extended-architecture-use-cases-and-leveraging-soreuseport https://www.cybertec-postgresql.com/en/postgresql-understanding-deadlocks/ https://www.enterprisedb.com/postgres-tutorials/introduction-postgresql-performance-tuning-and-optimization https://yugonagata-pgsql.blogspot.com/2021/04/implementing-incremental-view_61.html https://blog.crunchydata.com/blog/choice-of-table-column-types-and-order-when-migrating-to-postgresql https://www.migops.com/blog/2021/04/25/char1-to-boolean-datatype-transformation-while-migrating-to-postgresql/ https://akorotkov.github.io/blog/2021/04/30/arm/ https://arctyp

In this episode of Scaling Postgres, we discuss implementing row level triggers, how to return dynamic columns using pl/pgsql, and when to use citext. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://pganalyze.com/blog/custom-postgres-data-types-ruby-rails https://www.cybertec-postgresql.com/en/what-to-return-from-a-postgresql-row-level-trigger/ https://www.depesz.com/2021/04/21/getting-value-from-dynamic-column-in-pl-pgsql-triggers/ https://www.youtube.com/watch?v=iq8sUHYilPk https://www.highgo.ca/2021/04/22/hash-index-internals/ https://www.highgo.ca/2021/04/19/calling-stored-procedure-from-java-part-2/ https://pgsqlpgpool.blogspot.com/2021/04/visibility-with-query-cache.html https://postgresql.life/post/pavlo_golub/

In this episode of Scaling Postgres, we discuss 5 years of scaling lessons, working with synchronous commit, managing connections and cleaning up BLOBs. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://onesignal.com/blog/lessons-learned-from-5-years-of-scaling-postgresql/ https://momjian.us/main/blogs/pgblog/2021.html#April_14_2021 https://blog.crunchydata.com/blog/your-guide-to-connection-management-in-postgres https://www.cybertec-postgresql.com/en/cleaning-up-large-number-blobs/ https://www.keithf4.com/managing-transaction-id-exhaustion-wraparound-in-postgresql/ https://blog.crunchydata.com/blog/postgres-is-out-of-disk-and-how-to-recover-the-dos-and-donts https://www.cybertec-postgresql.com/en/finally-a-system-level-read-all-data-role-for-postgresql/ https://www.citusdata.com/blog/2021/04/10/talk-at-cmu-how-citus-distributes-postgresql-via-extension-apis/ https://www.enterprisedb.com/blog/benchmarking-postgresql-setting-ra

In this episode of Scaling Postgres, we discuss tips for using psql, stories from a Postgres 13 upgrade, exploring the role hierarchy and replica scaling considerations. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://mydbanotebook.org/psql_tips_all.html https://blog.discourse.org/2021/04/standing-on-the-shoulders-of-a-giant-elephant/ https://www.cybertec-postgresql.com/en/postgresql-get-member-roles-and-permissions/ https://momjian.us/main/blogs/pgblog/2021.html#April_9_2021 https://www.migops.com/blog/2021/04/09/pgbackrest-the-best-postgres-backup-tool-with-a-very-active-community/ https://blog.crunchydata.com/blog/announcing-google-cloud-storage-gcs-support-for-pgbackrest https://blog.crunchydata.com/blog/introducing-pgbackrest-multiple-repository-support https://digitalis.io/blog/postgresql/part1-postgresql-ha-patroni-etcd-haproxy/ https://digitalis.io/blog/postgresql/deploying-postgresql-for-high-availability-with-patron