efficiently trimming postgresql tables

Posted by agilefall on Stack Overflow See other posts from Stack Overflow or by agilefall
Published on 2010-04-19T23:16:23Z Indexed on 2010/04/19 23:23 UTC
Read the original article Hit count: 194

Filed under:

postgresql

|

foreign-keys

|

efficiency

I have about 10 tables with over 2 million records and one with 30 million. I would like to efficiently remove older data from each of these tables.

My general algorithm is:

create a temp table for each large table and populate it with newer data
truncate the original tables
copy tmp data back to original tables using: "insert into originaltable (select * from tmp_table)"

However, the last step of copying the data back is taking longer than I'd like. I thought about deleting the original tables and making the temp tables "permanent", but I lose constraint/foreign key info.

If I delete from the tables directly, it takes much longer. Given that I need to preserve all foreign keys and constraints, are there any faster ways of removing the older data?

Thanks.

© Stack Overflow or respective owner

Related posts about postgresql

Postgresql fails to start on Ubuntu 10.04.4 LTS

as seen on Ask Ubuntu - Search for 'Ask Ubuntu'
I installed postgresql 9.2 from add-apt-repository ppa:pitti/postgresql using apt-get install postgresql-9.2 At the end of the install and every time I try to launch postgresql by using the following command /etc/init.d/postgresql start or service postgresql start I get this error: Error:… >>> More
can't install psycopg2 in my env on mac os x lion

as seen on Server Fault - Search for 'Server Fault'
I tried install psycopg2 via pip in my virtual env, but got this error: ld: library not found for -lpq (full log here: http://pastebin.com/XdmGyJ4u ) I tried install postgres 9.1 from .dmg and via port, (gksks)iMac-Alexander:~ lorddaedra$ locate libpq /Developer/SDKs/MacOSX10.7.sdk/usr/include/libpq /Developer/SDKs/MacOSX10… >>> More
Postgresql has broken apt-get on Ubuntu

as seen on Super User - Search for 'Super User'
On ubuntu 12.04, whenever I try to install a package using apt-get I'm greeted by: The following packages have unmet dependencies: postgresql-9.1 : Depends: postgresql-client-9.1 but it is not going to be instal led E: Unmet dependencies. Try 'apt-get -f install' with no packages (or specify a so lution)… >>> More
Installing PostgreSQL on FreeBSD (with ports)

as seen on Server Fault - Search for 'Server Fault'
Hey everyone, I am trying to install (using ports) PostgreSQL on a virtual server, running FreeBSD. My one question is this: Which of the following should I install? postgresql-contrib postgresql-docs postgresql-jdbc postgresql-libpgeasy postgresql-libpq++ postgresql-libpqxx postgresql-odbc … >>> More
Strange permission errors in new PostgreSQL installation

as seen on Server Fault - Search for 'Server Fault'
A freshly installed PostgreSQL (with configuration overwritten) won't start: $ sudo service postgresql start * Starting PostgreSQL 9.1 database server * Error: could not read /etc/postgresql/9.1/main/postgresql.conf: Permission denied Looks like it should be able to read it though: $ ls -l postgresql… >>> More

Related posts about foreign-keys

Foreign keys in django admin list display

as seen on Stack Overflow - Search for 'Stack Overflow'
If a django model contains a foreign key field, and if that field is shown in list mode, then it shows up as text, instead of displaying a link to the foreign object. Is it possible to automatically display all foreign keys as links instead of flat text? (of course it is possible to do that on a… >>> More
Multiple foreign keys in one table to 1 other table in mysql

as seen on Stack Overflow - Search for 'Stack Overflow'
Hey guys, I got 2 tables in my database: user and call. User exists of 3 fields: id, name, number and call : id, 'source', 'destination', 'referred', date. I need to monitor calls in my app. The 3 ' ' fields above are actually userid numbers. now i'm wondering, can i make those 3 field foreign… >>> More
Create automatically table that is composed by its primery key and 2 other foreign keys

as seen on Stack Overflow - Search for 'Stack Overflow'
I have table A with a primary id, a table B with a primary key id also and another table C with a primary key id and rows Aid and Bid where Aid and Bid are foreign keys of C and the primary keys of A and B respectively. One way of populating these DB would be populating table A, table B and table… >>> More
Foreign keys with Rails' ActiveRecord::Migration?

as seen on Stack Overflow - Search for 'Stack Overflow'
Hello, I'm new to Ruby on Rails (I know Ruby just decently though) and looking at the Migration tools, it sounds really awesome. Database schemas can finally (easily) go in source control. Now my problem with it. When using Postgres as the database, it does not setup foreign keys. I would like the… >>> More
Working with foreigh keys - cannot insert

as seen on Stack Overflow - Search for 'Stack Overflow'
Hi everyone! Doing my first tryouts with foreign keys in a mySQL database and are trying to do a insert, that fails for this reason: Integrity constraint violation: 1452 Cannot add or update a child row: a foreign key constraint fails Does this mean that foreign keys restrict INSERTS as well as… >>> More