How to Receive Image Bytes From HTTP Response Through C Socket Programming?

[shinsengumi]

Hi,

What command/technique is needed in a C program to receive image bytes through socket programming from an HTTP response? I want to develop a program that requests for an image from a server, then upon receiving the bytes comprising the image, it should write these bytes into a newly created file such that an image file is created. The image is of .jpg format. I really don't know how to create a program that receives and stores bytes through sockets so I hope someone here knows how to do it. So far, what I know is how to receive arrays of characters (strings) but now I want to know how to receive image bytes.

Also, the response my program is receiving has an HTTP header that comes before the actual image bytes so I need to remove the HTTP header first before writing the image bytes into a file. How do I do it? Thanks in advance. I'm working on a Windows XP machine.

The following is a code snippet of my program (so far, this is not yet able to create a .jpg image)

  char recvbuf[MAXBUFLEN];
  FILE *fpic;
 
  fpic = fopen("picture.jpg", "w");
  if (!fpic) {
    printf("Error writing file\n");
    return 1;
  }
 
  numbytes=recv(ConnectSocket,recvbuf,sizeof(recvbuf),0);
  if (numbytes == SOCKET_ERROR)
     printf("Server: recv() error %ld.\n", WSAGetLastError());
 
  else {
       printf("Server: Received data is: \"%s\"\n", recvbuf);
       printf("Server: Bytes received: %ld.\n", numbytes);
       fwrite(recvbuf, 1, strlen(recvbuf), fpic);
       fclose(fpic);
  }
 

[Banfa]

I am guessing that you are expecting recv to return the entire entire HTTP packet containing the image in the 1 call. However that is not how TCP sockets work, if the socket has received data then when you call recv you can at best guarantee that you will receive 1 byte and for large images it is likely that you will need to call recv multiple times.

I suggest you get hold of and read Beej's Guide to Network Programming.

HTTP is a fairly simple protocol when returning an image it just sends its header and then takes the image data onto the end of the message (in binary) you just have to read the data off and store it.

HTTP uses caridge return as a separator. So for example you can attach to this site (using a telnet program) and request the main site image and what you get returned is

HTTP/1.1 200 OK
Date: Mon, 07 Mar 2011 12:58:39 GMT
Server: LiteSpeed
Accept-Ranges: bytes
Connection: Keep-Alive
Keep-Alive: timeout=5, max=100
Last-Modified: Wed, 06 Oct 2010 09:38:11 GMT
Content-Type: image/gif
Content-Length: 744
Cache-Control: max-age=2592000
Expires: Wed, 06 Apr 2011 12:58:39 GMT
 
GIF89ax▲─???▀Ù㻽½½{{KKK*╣8ÀÊ~╠╠╠þþþfffÖÖÖfff▄▄▄¶¶¶WWW───ç┤)¤ß¬»═qþ*ıÃ▄øƒ├SÀÀÀççç┐Îì!¨♦,x▲!Ädi×h¬«lÙ¥p,¤tm▀x«´|´áPà©Å╚d)
☺h:ƒÉ├!z@×Ûëüep▒Ó«!LNî▲â┼ò╠0]♦*©|¥$â↨♥¾╣8ÝØ►_v`mäXf♠çm%osÆquìM}♠âMÅ$
                                                                    êÖûOåíf
                                                                           îì£#æôÆòí♠VXZ"ÜÄÁíNúûf↓í®"½¼r«Më©O↓"ù#Xü"
 
ÈÈ♥X↓ıÈ
       c╔┌▀yºù▀ı▄ÞÞ↨ÚýÞ¶t$╠Ã"XÿáO♥"×O¶┼À&─¿└ÆG┼£)☻└¤▼▬☻gÍ4)á└▬░↨╗ÚRíÇÓèâ        ↨.¹4┬▬,å÷─êûB^Fü)áaÓå☻$
                                                                                                       à─ÔÕ║WÆÉ?.O@[@┤(Ð
♦┬@ └└ƒ═8)Ú┤─á  Ya2b¶Ju$í☻↓h>5§╬?;
                                  ~Û    ├│%L↕┐F`hT éü▒%╩
Ïuä☻__§Ó%í¸▀▒T▬░zµ¡è┴%░ò█fÔA♦♠³└dX<ó±T'■õ¨à+╩fͦ♦☻iªQ÷#∟;sk┴ûÙ²ªüÏ╔∟
                                                                    ☺Æ+WnAõ┌¦Wâk▄õZ║¶↨Âåi↨0┴±_Ù├W╝▲q¢à☻┌
                                                                                                        Â│Û■/â↨/┤øðéM↔xx
PîÛçÑ▀hÿÐÛ╔!A~õ☻@[°§&▲ex]↓¿XÇp
6▓Ç]+h►çä)T¶ÌìMQË☻ÄjØ`Ò{jß(õìfêé<äÓüBy[±å╩tç└ÆDHÓäû,░Üå▒0×p
                              9Ç6ö® ú4ßõô%╚Ï$öT:®┴öUfi;
 

You get the headers, which give the content type and length, each header ends in a CR and then there is an additional CR at the end of the headers giving a blank line and then the image content (binary GIF data).

[shinsengumi]

So, what do I have to do now? How do I remove the header? Should I try to find the \r\n\r\n part? Also, how do I know if I have already received all the data that the server had sent to me? Thanks

[Banfa]

You read the headers by simply reading the data line by line. You look for the start of content (the \r\n\r\n) and you know how long the content is because it is in the content-length header that you will already have parsed.

You will have to accept that you are going to start receiving data without knowing how much data you are finally going to receive but that you will know how much data you are receiving before you have finished receiving it.

[shinsengumi]

>you know how long the content is because it is in the content-length header that you will already have parsed

I know the content length of the data, but how will my program know it from the header? Do I have to create a function that will find the "Content-Length" part of the header?

[Banfa]

At the very least you should write code that parses the "Content-Length" header. It should be too hard as all you need to do is look for "Content-Length" at the start of any header line. It may be worth looking for the "Content-Type" header too just to verify what you are downloading, you can probably ignore the rest of the headers.

The content length header is well defined so once you have located it and skipped passed the ":" a quick call to atoi should get you the length.