Fishing for a way to search a humongous database

[richard]

http://safer.fmcsa.dot.gov/CompanySnapshot.aspx
How could I go about automating a search of this site's database?
I'm not interested in all the details they present on the output form, just
the bare basics.
That is, trucking company name, city location, and zip code and a couple of
other fields
With over a million records, manual searching is out of the question.
Every trucking company is required to have, and display, a USDOT number.
Using that field as a starting point, one might be able to extract the rest
of the information.
Going by a name search, the site only returns 500 matches and does not care
where in the name the field appears.

I'm open to suggestions.

[Paul Lautman]

richard wrote:

http://safer.fmcsa.dot.gov/CompanySnapshot.aspx
How could I go about automating a search of this site's database?
I'm not interested in all the details they present on the output
form, just the bare basics.
That is, trucking company name, city location, and zip code and a
couple of other fields
With over a million records, manual searching is out of the question.
Every trucking company is required to have, and display, a USDOT
number. Using that field as a starting point, one might be able to
extract the rest of the information.
Going by a name search, the site only returns 500 matches and does
not care where in the name the field appears.

I'm open to suggestions.

Create a fulltext index?

[richard]

"Paul Lautman" <pa**********@btinternet.comwrote in message
news:4l************@individual.net...

richard wrote:

>http://safer.fmcsa.dot.gov/CompanySnapshot.aspx
How could I go about automating a search of this site's database?
I'm not interested in all the details they present on the output
form, just the bare basics.
That is, trucking company name, city location, and zip code and a
couple of other fields
With over a million records, manual searching is out of the question.
Every trucking company is required to have, and display, a USDOT
number. Using that field as a starting point, one might be able to
extract the rest of the information.
Going by a name search, the site only returns 500 matches and does
not care where in the name the field appears.

I'm open to suggestions.

Create a fulltext index?

I have no clues as to what that is.

[richard]

"richard" <do*@john.sonwrote in message
news:ec*********@news2.newsguy.com...

http://safer.fmcsa.dot.gov/CompanySnapshot.aspx
How could I go about automating a search of this site's database?
I'm not interested in all the details they present on the output form,
just the bare basics.
That is, trucking company name, city location, and zip code and a couple
of other fields
With over a million records, manual searching is out of the question.
Every trucking company is required to have, and display, a USDOT number.
Using that field as a starting point, one might be able to extract the
rest of the information.
Going by a name search, the site only returns 500 matches and does not
care where in the name the field appears.

I'm open to suggestions.

PS:

One of my ideas was to automate a search simulating a manual search using
the usdot number as a base.
If the number was issued, then save the output page as a text file.
If it was not issued, skip it and go to the next. As an unissued number
results in a stock "error" message.

[Jerry Stuckle]

richard wrote:

http://safer.fmcsa.dot.gov/CompanySnapshot.aspx
How could I go about automating a search of this site's database?
I'm not interested in all the details they present on the output form, just
the bare basics.
That is, trucking company name, city location, and zip code and a couple of
other fields
With over a million records, manual searching is out of the question.
Every trucking company is required to have, and display, a USDOT number.
Using that field as a starting point, one might be able to extract the rest
of the information.
Going by a name search, the site only returns 500 matches and does not care
where in the name the field appears.

I'm open to suggestions.

Get a copy of the database from FMCSA?

--
==================
Remove the "x" from my email address
Jerry Stuckle
JDS Computer Training Corp.
js*******@attglobal.net
==================

[richard]

"Jerry Stuckle" <js*******@attglobal.netwrote in message
news:_t******************************@comcast.com. ..

richard wrote:

>http://safer.fmcsa.dot.gov/CompanySnapshot.aspx
How could I go about automating a search of this site's database?
I'm not interested in all the details they present on the output form,
just the bare basics.
That is, trucking company name, city location, and zip code and a couple
of other fields
With over a million records, manual searching is out of the question.
Every trucking company is required to have, and display, a USDOT number.
Using that field as a starting point, one might be able to extract the
rest of the information.
Going by a name search, the site only returns 500 matches and does not
care where in the name the field appears.

I'm open to suggestions.

Get a copy of the database from FMCSA?

Have asked, no reply yet.

[Jerry Stuckle]

richard wrote:

"Jerry Stuckle" <js*******@attglobal.netwrote in message
news:_t******************************@comcast.com. ..

>>richard wrote:

>>>http://safer.fmcsa.dot.gov/CompanySnapshot.aspx
How could I go about automating a search of this site's database?
I'm not interested in all the details they present on the output form,
just the bare basics.
That is, trucking company name, city location, and zip code and a couple
of other fields
With over a million records, manual searching is out of the question.
Every trucking company is required to have, and display, a USDOT number.
Using that field as a starting point, one might be able to extract the
rest of the information.
Going by a name search, the site only returns 500 matches and does not
care where in the name the field appears.

I'm open to suggestions.

Get a copy of the database from FMCSA?

Have asked, no reply yet.

You may have to dig around to find the right person(s) to ask. If it's
like other federal agencies, people just ignore questions not in their
job function. And if it is in their job function, they may or may not
answer - depending on whether the coin comes up heads or tails :-)

--
==================
Remove the "x" from my email address
Jerry Stuckle
JDS Computer Training Corp.
js*******@attglobal.net
==================