469,323 Members | 1,210 Online
Bytes | Developer Community
New Post

Home Posts Topics Members FAQ

Post your question to a community of 469,323 developers. It's quick & easy.

How to send file with WebService?

ad
I want to send a DataSet to WebService, but the DataSet if too huge(there
about 50000 records, and 50 fields every record).
My solution is
1.save the DataSet as XML file,
2.zip the XML file.
3. send the zip file to WebService.
4. Unzip the zip file to XML.
5. Load XML file into DataSet.
6.Bulk copy the XML file to DataBase.

I can do the Steps 1,2, but I do'nt know how to set the zip file to
WebService in Step 3.

How can I do that?


Nov 17 '05 #1
4 18183
Hi ad,
fiftythousand records with fifty fields each, is is a lot. But too much?
Before you try to do the xml zipping and uploading, have a look at the
webservice.timeout property. Maybee this one will help to overcome your
problems.

Here is som emor information:
http://www.kleinurl.de/?ks9sdi89

greetings from germany
Chris

Nov 17 '05 #2
Ste
Presuming you are sending the file over the wire via soap/http

Are you sending the file as a soap attachment?

If not i presume you are tyring to embed the file in the soap body - to do
this you will have to convert the file bytes into Base64 and embed the
base64 encoded zip file in the soap body.. have you done this... PS this
will make your zip file bigger.

eg..
1.save the DataSet as XML file,
2.zip the XML file.
2a base64 zip file to get a human readable file encoded
2b embedd the base64 bytes in the soap doc
3. send the zip file to WebService.
4. Unzip the zip file to XML.
5. Load XML file into DataSet.
6.Bulk copy the XML file to DataBase.


To save file size - do you really need to save the dataset as xml... have
you thought about comma delimited fields, run length encoding.. before
zip'ing etc.. The data file size can be massively reduced.. and as this data
transfer of such a big file will be your bottleneck.. the amount of time you
spend processing the file back to xml on arrival can often be offset.

Alternatively design your solution to send the data in chunks...

Nov 17 '05 #3
ad
Thanks a lot!
Could you tell me more detail?
1. What about " send the data in chunks" How to implement ?
2. ....about comma delimited fields, run length encoding.. HOw to run
lenght encoding?
3. How to embed base64 encoded into the soap body?


"Ste" <no*****@nospam.com> 撰寫於郵件新聞:gv********************@pipex.net...
Presuming you are sending the file over the wire via soap/http

Are you sending the file as a soap attachment?

If not i presume you are tyring to embed the file in the soap body - to do
this you will have to convert the file bytes into Base64 and embed the
base64 encoded zip file in the soap body.. have you done this... PS this
will make your zip file bigger.

eg..
1.save the DataSet as XML file,
2.zip the XML file.


2a base64 zip file to get a human readable file encoded
2b embedd the base64 bytes in the soap doc
3. send the zip file to WebService.
4. Unzip the zip file to XML.
5. Load XML file into DataSet.
6.Bulk copy the XML file to DataBase.


To save file size - do you really need to save the dataset as xml... have
you thought about comma delimited fields, run length encoding.. before
zip'ing etc.. The data file size can be massively reduced.. and as this
data transfer of such a big file will be your bottleneck.. the amount of
time you spend processing the file back to xml on arrival can often be
offset.

Alternatively design your solution to send the data in chunks...


Nov 17 '05 #4
Ste
1) Sending in chunks.. [not talking about raw http encoding here] .. You
could make your web service send subsets of the records eg the client makes
multiple calls to your webservice sending a soap payload along the lines
of..

The server receives the data in pieces, and processes each record in
turn...More network transfer, but if things go wrong - you only need to
resend the last chunk you sent.. not the whole dataset

<recordblocktype>start</recordblocktype>
<totalrecords>10</totalrecords>
<recordstart>1</recordstart>
<recordend>5</recordend>
<datarows> record_01 data
record_02 data
record_03 data
record_04 data
record_05 data
</datarows>

<recordblocktype>chunk</recordblocktype>
...
...

<recordblocktype>end</recordblocktype>
<totalrecords>20</totalrecords>
<recordstart>6</recordstart>
<recordend>10</recordend>
<datarows>
record_06 data
record_07 data
record_08 data
record_09 data
record_10 data
</datarows>

2) You are taking xml and compressing it into zip format..dont compress the
xml version - convert to the data into another format before you compress
it.. for example..

(this trivial list of individual data rows can easily be represented in a
form other that xml)
xml version..
<data_1>somedata1</data_1><data_2>somedata2</data_2><data_3>somedata3</data_3><data_4>somedata4</data_4>
comma del.. somedata1,somedata2,somedata3,somedata4

As you can see before compression the comma del version of the data is
already smaller.. this should give you a bettter compression ration.

You will have to find a suitable compromise here between ease of use and
compression.

3) Look up the [!CDATA] tag for soap .. read this synpsis

http://webservices.xml.com/pub/a/ws/...endpoints.html

HTH

Steve

"ad" <fl****@wfes.tcc.edu.tw> wrote in message
news:%2****************@TK2MSFTNGP12.phx.gbl...
Thanks a lot!
Could you tell me more detail?
1. What about " send the data in chunks" How to implement ?
2. ....about comma delimited fields, run length encoding.. HOw to run
lenght encoding?
3. How to embed base64 encoded into the soap body?


"Ste" <no*****@nospam.com>
撰寫於郵件新聞:gv********************@pipex.net...
Presuming you are sending the file over the wire via soap/http

Are you sending the file as a soap attachment?

If not i presume you are tyring to embed the file in the soap body - to
do this you will have to convert the file bytes into Base64 and embed the
base64 encoded zip file in the soap body.. have you done this... PS this
will make your zip file bigger.

eg..
1.save the DataSet as XML file,
2.zip the XML file.


2a base64 zip file to get a human readable file encoded
2b embedd the base64 bytes in the soap doc
3. send the zip file to WebService.
4. Unzip the zip file to XML.
5. Load XML file into DataSet.
6.Bulk copy the XML file to DataBase.


To save file size - do you really need to save the dataset as xml... have
you thought about comma delimited fields, run length encoding.. before
zip'ing etc.. The data file size can be massively reduced.. and as this
data transfer of such a big file will be your bottleneck.. the amount of
time you spend processing the file back to xml on arrival can often be
offset.

Alternatively design your solution to send the data in chunks...



Nov 17 '05 #5

This discussion thread is closed

Replies have been disabled for this discussion.

Similar topics

3 posts views Thread by Robb Gilmore | last post: by
10 posts views Thread by John Davies | last post: by
3 posts views Thread by Code Monkey | last post: by
5 posts views Thread by tommaso.gastaldi | last post: by
reply views Thread by ultradiv | last post: by
reply views Thread by Gurmeet2796 | last post: by
reply views Thread by mdpf | last post: by
reply views Thread by harlem98 | last post: by
reply views Thread by listenups61195 | last post: by
By using this site, you agree to our Privacy Policy and Terms of Use.