RailsPgAdapter

This project allows you to monkey patch ActiveRecord (PostgreSQL) and auto-heal applications in production when PostgreSQL database fails over or when a cached column (in ActiveRecord schema cache) is removed from the database from a migration in another process.

Installation

Install the gem and add to the application's Gemfile by executing:

$ bundle add rails-pg-adapter

If bundler is not being used to manage dependencies, install the gem by executing:

$ gem install rails-pg-adapter

Usage

Auto healing connections when PostgreSQL database fails over

# config/initializers/rails_pg_adapter.rb

RailsPgAdapter.configure do |c|
  c.add_failover_patch = true
end

This will add the monkey patch which resets the ActiveRecord connections in the connection pool when the database fails over. The patch will reset the connection and re-raise the error each time it detects that an exception related to a database failover is detected.

Retrying queries

When the database is failing you can retry queries that are not in a transaction. The gem will perform a back off retry in establishing the connection. Once the back off is reached and no connection is found, it will bubble up the exception. Otherwise, the query will be retried with a new connection.

It is an opt-in functionality. You can supply your own back off figures for retries (in seconds) as following:

# config/initializers/rails_pg_adapter.rb

RailsPgAdapter.configure do |c|
  c.add_failover_patch = true
  c.reconnect_with_backoff = [0.5, 1, 2, 4, 8, 16] # seconds
...
end

Refresh model column information on the fly after an existing column is dropped

# config/initializers/rails_pg_adapter.rb

RailsPgAdapter.configure do |c|
  c.add_reset_column_information_patch = true
end

This will clear the ActiveRecord schema cache and reset the ActiveRecord column information memoized on the model. The patch will reset the relevant information and re-raise the error each time it detects that an exception related to a dropped column is raised.

How does it work

During a database failover in production, the ActiveRecord connection pool can become exhausted as queries are made against the database during the failover process. This can leave the ActiveRecord connection pools with stale or bad connections, even after the database has successfully recovered. Recovering from this issue usually requires a rolling restart of the application processes or containers.

RailsPgAdapter addresses this problem by resetting the connection pool and re-raises the original exception from an ActiveRecord monkey patch. This allows the application to auto-heal from stale connections on its own (after database recovery) when performing queries for a new request, without requiring manual intervention.

Another issue with ActiveRecord queries is PG::UndefinedColumn, which occurs when an ActiveRecord model includes a SELECT query with the name of a column that has been dropped from a Rails migration. This can happen even if the column isn't being referenced anywhere in the code. It occurs when a model is using ignored_columns, which prompts ActiveRecord to perform a dedicated lookup of the allowed columns in a select, such as SELECT "users".name, "users".template_id....", instead of SELECT "users".*. When a column like template_id is dropped, PostgreSQL throws an undefined column error, which is bubbled up by ActiveRecord into PG::UndefinedColumn. Recovering from this issue also usually requires a rolling restart of the application processes or containers.

RailsPgAdapter solves this second issue by resetting the ActiveRecord schema cache and memoized model column information when it detects a PG::UndefinedColumn raised from a monkey patch. Resetting the column information forces ActiveRecord to refresh its schema cache by loading the table information from the database and no longer reference the dropped column for new queries, without requiring manual intervention.

Development

• Install ruby 3.0

\curl -sSL https://get.rvm.io | bash

rvm install 3.0.0

rvm use 3.0.0

• docker compose up -d - to spin up postgres locally
• bundle exec rspec to run the tests.
• You can also run bin/console for an interactive prompt.

To install this gem onto your local machine, run bundle exec rake install.

Contributing

Bug reports and pull requests are welcome on GitHub at https://github.com/tines/rails-pg-adapter. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the code of conduct.

License

The gem is available as open source under the terms of the MIT License.

Code of Conduct

Everyone interacting in the RailsPgAdapter project's codebases, issue trackers, chat rooms and mailing lists is expected to follow the code of conduct.

Releasing a new version

• Bump version in version.rb
• Update CHANGELOG.md
• Push the changes to main
• Run the release script with the new version ./bin/release.sh 0.2.0
  • Note: It will ask for MFA.
• Create a new release - https://github.com/tines/rails-pg-adapter/releases/new