Performances QA testing

From PostgreSQL Wiki

This page centralizes the efforts on performances QA testing: available hardware, available tools, continuous benchmarking effort...

The PostgreSQL Performance lab is being created to allow community members of the Open Source database PostgreSQL to have enterprise class hardware to test on.

The testing that will occur includes industry standard workloads such as OLTP, DSS and BI. Furthermore we will also use the hardware for other practical and customer oriented testing to improve scalability (processor utilization, i/o, load balancing, etc.) and managing large data sets (loading, backups, restores, replication, etc).

Contents 1 Donations 2 QA platforms 3 Tools 4 Ideas

Donations

For donation inquiries, please contact Josh Berkus <josh @t postgresql.org> and Joshua Drake <jdrake @t postgresql.org>.

QA platforms

• QA Platform hosted at Command Prompt - Portland, Oregon, USA
• QA Platform hosted at Open Wide (France)

Tools

• Former OSDL work: Database Test Suite and Web interface
• pgbench-tools from Greg Smith
• Bristlecone from Continuent
• Tsung load injector allows to define sessions (containing queries and thinktime, etc) and replay them with very high concurrency setup. Can use many loading nodes at a time, multi OS support (written in erlang, extensible in this language)
• Tsung Ploter plots several tsung runs onto the same graphs set, for easy comparing. Uses python and matplotlib.
• Tsung DBT2 Implementation (tsung module in erlang), WIP, to get published asap.

Ideas

• look into sysbench - it has some issues with locking on postgresql but at least read-only it seems to work fine

• collecting all the various small samples and testcases posted over the last few years on -performance, -hackers & -bugs and put them into a test set

• consider doing tests using pgbench -M (simple|extended|prepared) to catch regressions in one of those modes

• resurrect Jan Wiecks tpc-w implementation available on pgfoundry

• add full text search benchmarking by using ftsbench from teodor

• XML benchmarking ?

• investigate QuickCheck and http://advogato.org/person/nconway/diary/23.html

• Implement the Star Schema Benchmark.