RUBiS is an auction site prototype modeled after eBay.com that is used to evaluate application design patterns and application servers performance scalability.

Our auction site benchmark implements the core functionality of an auction site: selling, browsing and bidding. We do not implement complementary services like instant messaging or newsgroups. We distinguish between three kinds of user sessions: visitor, buyer, and seller. For a visitor session, users need not register but are only allowed to browse. Buyer and seller sessions require registration. In addition to the functionality provided during visitor sessions, during a buyer session users can bid on items and consult a summary of their current bids, rating and comments left by other users. Seller sessions require a fee before a user is allowed to put up an item for sale. An auction starts immediately and lasts typically for no more than a week. The seller can specify a reserve (minimum) price for an item.

RUBiS is a free, open source initiative.

Several versions of RUBiS are implemented using three different technologies: PHP, Java servlets and EJB (Enterprise Java Bean).

In PHP and Java servlets, the application programmer is responsible for writing the SQL queries. In order to arrive at a fair comparison, we use exactly the same queries to the database in both environments.

An EJB server provides a number of services such as database access (JDBC), transactions (JTA), messaging (JMS), naming (JNDI) and management support (JMX). The EJB server manages one or more EJB containers. The container is responsible for providing component pooling and lifecycle management, client session management, database connection pooling, persistence, transaction management, authentication and access control.EJB containers automatically manage bean persistence, relieving the programmer of writing SQL code. We use these facilities to implement various EJB versions of RUBiS.

While using EJB 1.1 or 2.0 CMP model, we use session beans for complex queries that cannot be executed using CMP.

The Java servlets are used only as the presentation tier, to generate the HTML reply from the information retrieved from the bean.

RUBiS can be used from a web browser for testing purposes or with the provided benchmarking tool. We designed a client that emulates users behavior for variours workload patterns and provides statistics.

Our auction site defines 26 interactions that can be performed from the client’s Web browser. Among the most important ones are browsing items by category or region, bidding, buying or selling items, leaving comments on other users and consulting one’s own user page (known as myEbay on eBay). Browsing items also includes consulting the bid history and the seller’s information. We define two workload mixes: a browsing mix made up of only read-only interactions and a bidding mix that includes 15% read-write interactions. The bidding mix is the most representative of an auction site workload.

We implement a client-browser emulator. A session is a sequence of interactions for the same customer. For each customer session, the client emulator opens a persistent HTTP connection to the Web server and closes it at the end of the session. Each emulated client waits for a certain think time before initiating the next interaction. The next interaction is determined by a state transition matrix that specifies the probability to go from one interaction to another one.

The think time and session time for all benchmarks are generated from a negative exponential distribution with a mean of 7 seconds and 15 minutes, respectively. We vary the load on the site by varying the number of clients.

We use a MySQL database that contains 7 tables : users, items, categories, regions, bids, buy_now and comments. The users table records contain the user’s name, nickname, password, region, rating and balance. Besides the category and the seller’s nickname, the items table contains the name that briefly describes the item and a more extensive description, usually an HTML file. Every bid is stored in the bids table, which includes the seller, the bid, and a max_bid value used by the proxy bidder (a tool that bids automatically on behalf of a user). Items that are directly bought without any auction are stored in the buy_now table. The comments table records comments from one user about another. As an optimization, the number of bids and the amount of the current maximum bid are stored with each item to prevent many expensive lookups of the bids table. This redundant information is necessary to keep an acceptable response time for browsing requests. As users only browse and bid on items that are currently for sale, we split the item table in a new and an old item table. The very vast majority of the requests access the new items table, thus considerably reducing the working set used by the database.

We sized our system according to some observations found on the eBay Web site. We always have about 33,000 items for sale, distributed among eBay’s 20 categories and 62 regions. We keep a history of 500,000 auctions in the old-items table. There is an average of 10 bids per item, or 330,000 entries in the bids table. The buy_now table is small, because less than 10% of the items are sold without auction. The users table has 1 million entries. We assume that users give feedback (comments) for 95% of the transactions. The comments table contains about 506,500 comments refering either to items or old items. The total size of the database, including indices, is 1.4GB.

Learn more about RUBiS in the documentation section and download the code or the experimental results.

Any input or personal view for improving and/or developing RUBiS is welcome. RUBiS, as an open source project, also welcomes external contributions. Interested? Contact us.