any DBAs in the house?

jaguar · 04-06-2006, 05:09 PM

A friend and I were scoping how to do a project, we ended up with a situation where we would have around 10,000,40,000 records that would have 5 variables each having to be updated every 5-10 minutes . How can you handle that kind of situation without not needing a quad-opteron? temporary tables?

mbpark · 04-06-2006, 05:31 PM

Jag,

#1. What kind of database do you plan on using?
#2. What kind of API do you plan on using to connect to the database?
#3. I've done this before. If you bulk load the data into a permanent table and then use a UPDATE x SET y = (SELECT z from aa where aa.bb = x.cc) in Oracle, or UPDATE x SET y = aa.z FROM aa WHERE x.cc = aa.bb in SQL Server
you would be able to pull this off with a single processor very quickly.

With SQL Server, it would not be hard to do. Same with Oracle or Postgres. You're not doing this kind of work with mySQL (good for inserts and deletes, but not updates from what I know, this is an involved subselect query).

PM me if you need more info

.

Mitch

Happy Monkey · 04-06-2006, 05:36 PM

Quote:

Originally Posted by jaguar

...we ended up with a situation where we would have around 10,000,40,000 records...

Are you missing a digit or a dash here?

jaguar · 04-06-2006, 06:34 PM

pm shall be done.

04-06-2006, 06:34 PM	#4
jaguar whig Join Date: Apr 2001 Posts: 5,075	pm shall be done. __________________ Good friends, good books and a sleepy conscience: this is the ideal life. - Twain

04-06-2006, 05:09 PM	#1
jaguar whig Join Date: Apr 2001 Posts: 5,075	any DBAs in the house? A friend and I were scoping how to do a project, we ended up with a situation where we would have around 10,000,40,000 records that would have 5 variables each having to be updated every 5-10 minutes . How can you handle that kind of situation without not needing a quad-opteron? temporary tables? __________________ Good friends, good books and a sleepy conscience: this is the ideal life. - Twain

04-06-2006, 05:31 PM	#2
mbpark Lecturer Join Date: Jan 2001 Location: Carmel, Indiana Posts: 761	Database Jag, #1. What kind of database do you plan on using? #2. What kind of API do you plan on using to connect to the database? #3. I've done this before. If you bulk load the data into a permanent table and then use a UPDATE x SET y = (SELECT z from aa where aa.bb = x.cc) in Oracle, or UPDATE x SET y = aa.z FROM aa WHERE x.cc = aa.bb in SQL Server you would be able to pull this off with a single processor very quickly. With SQL Server, it would not be hard to do. Same with Oracle or Postgres. You're not doing this kind of work with mySQL (good for inserts and deletes, but not updates from what I know, this is an involved subselect query). PM me if you need more info . Mitch

Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)