I have this giant table in our database that represents a dumping ground for
file imports. It's 44 columns wide and nominally about 30 million records,
indexed in blocks by the file the records came from.
I also have this incredibly elaborate class library that cleans the data in
various columns (you would not believe the trash that comes in on the
PhoneNumber field)
One of the things that has been bugging me for a long time is this issue
here:
In my cleaner app, I have dozens of methods that do really heavy string
manipulation that all pretty much go like this:
1. Get a DataTable from the raw-data dumping-ground table via a
SqlDataAdapter
2. Loop over the Rows collection cleaning the values based on business rules
3. Put the values into the actual entity tables where they belong, via a
SqlDataAdapter
But here's the whammy:
Since the DataTable was populated from a DataAdapter, the DataRows all think
they came from the database, and rightly so. But after my cleaning
algorithms, the cleaned values represents INSERTs to their respective
tables. But when I say da.Update(dt), the DataAdapter sees a DataTable with
*modified* values, not *new* values, and instinctively runs its
UpdateCommand. But I'm not doing updates, I'm doing inserts. So I keep
finding myself having to LIE to the DataAdapter by putting a stored
procedure that does INSERTs into the DataAdapter's UpdateCommand.
It works, but I don't like it. It seems like a perverse filthy hack.
Anyone have a better idea?
--
Peace & happy computing,
Mike Labosh, MCSD
"Mr. McKittrick, after very careful consideration, I have
come to the conclusion that this new system SUCKS."
-- General Barringer, "War Games"