0
nigel99

MySQL and PHP help

Recommended Posts

Can anyone help with a MySQL and PHP query?

I have a PHP script that takes a data stream and after parsing it inserts the results into a MySQL table. We need to check whether the data is unique based on an address and timestamp combination, so my current solution is to query the database with the timestamp and address and if a match is found discard the duplicate data. The problem is that this only works where the data has a delay between duplicate events. It is like there is some sort of write buffer that causes the query to return false when it should be true...

Any idea's welcome:)
Experienced jumper - someone who has made mistakes more often than I have and lived.

Share this post


Link to post
Share on other sites
Quote

Can anyone help with a MySQL and PHP query?

I have a PHP script that takes a data stream and after parsing it inserts the results into a MySQL table. We need to check whether the data is unique based on an address and timestamp combination, so my current solution is to query the database with the timestamp and address and if a match is found discard the duplicate data. The problem is that this only works where the data has a delay between duplicate events. It is like there is some sort of write buffer that causes the query to return false when it should be true...

Any idea's welcome:)



Have it read the data into a seperate array, after every capture process (or the end of the loop) have it process the data in the second array.

You could also just write to php scripts as different pages and load them both in your browser, one to fill the second array, one to process the data into the first.
"I may be a dirty pirate hooker...but I'm not about to go stand on the corner." iluvtofly
DPH -7, TDS 578, Muff 5153, SCR 14890
I'm an asshole, and I approve this message

Share this post


Link to post
Share on other sites
Quote

Can anyone help with a MySQL and PHP query?

I have a PHP script that takes a data stream and after parsing it inserts the results into a MySQL table. We need to check whether the data is unique based on an address and timestamp combination, so my current solution is to query the database with the timestamp and address and if a match is found discard the duplicate data. The problem is that this only works where the data has a delay between duplicate events. It is like there is some sort of write buffer that causes the query to return false when it should be true...

Any idea's welcome:)



You could make a unique key from the two data fields.
You could do subsequent queries with the distinct modifier (and not worry about dupes)
You can run a dupe checker later and remove dupes.

.
.
Make It Happen
Parachute History
DiveMaker

Share this post


Link to post
Share on other sites
is the data in your php script "unique" to the script?

If so, create a unique key in the mysql database, and just insert the data and implement the error handling in the php script instead of checking the database first, and then doing an insert.. that is going to back up fast if you have high volume..

Share this post


Link to post
Share on other sites
thanks that sounds about right as an approach. We are in the process of trashing the PHP and moving across to C++ as PHP is a crap language for what we are doing anyway.

We should be able to tweak things to make the unique key from the two data fields.
Experienced jumper - someone who has made mistakes more often than I have and lived.

Share this post


Link to post
Share on other sites
Quote

thanks that sounds about right as an approach. We are in the process of trashing the PHP and moving across to C++ as PHP is a crap language for what we are doing anyway.

We should be able to tweak things to make the unique key from the two data fields.



I think some clarification might be in order.
I was suggesting adding a unique key to the mysql DB based on a combination of the two data fields.
(You have to make sure that the formatting of the combined field works for the timestamp format that you use, accounting for padding of spaces or 0)
The DB then processes the requests and ensures only one entry for the key.

I was not suggesting that you use the scripting language or a cgi language to process the determination of uniqueness.

Also, just because the two data fields that you are looking at match, does not mean the rest of the record is identical.

.
.
Make It Happen
Parachute History
DiveMaker

Share this post


Link to post
Share on other sites
Quote

Quote

thanks that sounds about right as an approach. We are in the process of trashing the PHP and moving across to C++ as PHP is a crap language for what we are doing anyway.

We should be able to tweak things to make the unique key from the two data fields.



I think some clarification might be in order.
I was suggesting adding a unique key to the mysql DB based on a combination of the two data fields.
(You have to make sure that the formatting of the combined field works for the timestamp format that you use, accounting for padding of spaces or 0)
The DB then processes the requests and ensures only one entry for the key.

I was not suggesting that you use the scripting language or a cgi language to process the determination of uniqueness.

Also, just because the two data fields that you are looking at match, does not mean the rest of the record is identical.

.



sorry my bad communications skills at work. I understood what you were saying and through in the extra noise about moving from PHP...

thanks
Experienced jumper - someone who has made mistakes more often than I have and lived.

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

0