Hi,
I am having some probs with copying memory blocks around (part of a
messaging library) and I would like some confirmation to make sure that
I'm going about things the right way.
I have some data types defined thus:
typedef enum {
ONE ,
TWO
} EnumOne ;
typedef enum {
VAL_LONG ,
VAL_DOUBLE ,
VAL_STRING ,
VAL_DATASET
}DataType ;
typedef union {
long lval ;
double fval ;
char* sval ;
void* ptr ;
} Value ;
typedef struct {
/* Header */
int x ;
int y ;
int z ;
char* s1 ; /* Always set to NULL for this discussion */
char* s2 ; /* Always set to NULL for this discussion */
/* Payload */
EnumOne e1 ;
DataType e2 ;
Value v ;
char *s3 ;
size_t size ;
}MyStruct ;
I have two questions:
1). Is my calculation of the number of bytes used by MyStruct correct?
MyStruct *ptr = NULL ;
....
/* calculate size */
ptr->size = 3*sizeof(int) + 2*sizeof(char*) +
sizeof(EnumOne) + sizeof(DataType )+ sizeof_value(va l,valType) +
((strlen(ptr->s3)+1)* sizeof(char)) + sizeof(size_t) ;
where function sizeof_value is defined as ff:
size_t sizeof_value(Va lue x, DataType y){
if (y == VAL_LONG) return sizeof(long) ;
else if (y == VAL_DOUBLE) return sizeof(double) ;
else if (y == VAL_STRING) return ((strlen(x.sval )+1))*sizeof(ch ar) ;
else return 0 ; //you're f***ed (only 3 types supported for now)
}
Assuming my size calcuulation above is correct, I will be able to copy
data from my (suitably populated) data structure to another variable,
using the value of the size element of the struct as the number of bytes
to copy argument of either memcpy or memmove - Right ?
i.e.
MyStruct *newBlock = calloc(1, sizeof(MyStruct ));
/* The line below can cause a crash if we oldBlock has a string value -
how do i overcome this ? */
memcpy( newBlock, oldBlock, oldBlock->size) ;
9  1955 
Hi Alfonso,
1) no you do not need to disect your struct and compute the sum of the
component sizes in this fashion ! you can just apply sizeof() to
MyStruct and in fact it is better to do so since sizeof() includes any
padding etc. necessary to accommodate the structure components. Also I
see a strlen in there (?) your structure does not contain the strings
but merely pointers to them, it is the cost of these pointers that
contributes to the size of the structure rather than the length of the
strings.
2) you need to be clear by what you mean by "copy" a structure e.g., do
you want to "copy pointers" (after which both structures contains
pointers to the same string in memory) or do you want to "make copies
of pointers" (after which each structure has pointers to its own copies
of strings which could change over time) - in the former a memcpy of
size sizeof(MyStruct ) suffices, in the latter a component copy (perhaps
using strdup for char pointers) might be what you need.
hth,
ajm.
ajm wrote: Hi Alfonso,
1) no you do not need to disect your struct and compute the sum of the
component sizes in this fashion ! you can just apply sizeof() to
MyStruct and in fact it is better to do so since sizeof() includes any
padding etc. necessary to accommodate the structure components. Also I
see a strlen in there (?) your structure does not contain the strings
but merely pointers to them, it is the cost of these pointers that
contributes to the size of the structure rather than the length of the
strings.
Think about it. If one stucture contains a string (char*) of length
1024, and the another variable of the same data type contains a string
(char*) of length 16, sizeof (DataStructure) will not report any
difference in size between the two variables. This is patently wrong
(see below)
2) you need to be clear by what you mean by "copy" a structure e.g., do
you want to "copy pointers" (after which both structures contains
pointers to the same string in memory) or do you want to "make copies
of pointers" (after which each structure has pointers to its own copies
of strings which could change over time) - in the former a memcpy of
size sizeof(MyStruct ) suffices, in the latter a component copy (perhaps
using strdup for char pointers) might be what you need.
I may have not asked the question correctly. But put simply, I need to
make a copy of a nested structure. A 'proper' copy, so that the
structure can be sent to another process on a different machine - you
may recall that I DID say that I was writing a messaging library. It
would be the second type of copying that I am interested in since the
data is being sent accross process boundaries and any raw pointers would
be meaningless in another process space.
hth,
ajm.
Alfonso Morra wrote On 09/28/05 17:21,:
ajm wrote:
Hi Alfonso,
1) no you do not need to disect your struct and compute the sum of the
component sizes in this fashion ! you can just apply sizeof() to
MyStruct and in fact it is better to do so since sizeof() includes any
padding etc. necessary to accommodate the structure components. Also I
see a strlen in there (?) your structure does not contain the strings
but merely pointers to them, it is the cost of these pointers that
contributes to the size of the structure rather than the length of the
strings.
Think about it. If one stucture contains a string (char*) of length
1024, and the another variable of the same data type contains a string
(char*) of length 16, sizeof (DataStructure) will not report any
difference in size between the two variables. This is patently wrong
(see below)
I can't find anything called DataStructure in the
code you posted (see up-thread). Are you referring to
MyStruct? That seems likely, because the first question
you asked was 1). Is my calculation of the number of bytes used by
MyStruct correct?
For the usual C meaning of "number of bytes used" the
answer is "No." The size of MyStruct is sizeof(MyStruct ),
not the sum of the sizes of its elements plus the size of
other things in other places that some of its elements
happen to point at.
What you're really after, it seems, is not the size of
a MyStruct instance, but the number of bytes in a "serialized
version," that is, the length of some kind of encoding of
the data represented by a MyStruct and its dependents. This
latter count depends on the "on the wire" representation you
choose; if you like, it depends on your protocol.
2) you need to be clear by what you mean by "copy" a structure e.g., do
you want to "copy pointers" (after which both structures contains
pointers to the same string in memory) or do you want to "make copies
of pointers" (after which each structure has pointers to its own copies
of strings which could change over time) - in the former a memcpy of
size sizeof(MyStruct ) suffices, in the latter a component copy (perhaps
using strdup for char pointers) might be what you need.
I may have not asked the question correctly. But put simply, I need to
make a copy of a nested structure. A 'proper' copy, so that the
structure can be sent to another process on a different machine - you
may recall that I DID say that I was writing a messaging library. It
would be the second type of copying that I am interested in since the
data is being sent accross process boundaries and any raw pointers would
be meaningless in another process space.
Put simply, this is not what is usually called a "copy."
(Nor do I see anything in your code that I'd call a "nested
structure:" There's only one struct type shown, and it contains
no other struct types.)
You'll probably find it helpful to think not of "copies"
but of internal and external representations of your data.
The two machines may use entirely different representations
even if the MyStruct declarations are identical (the struct
padding may differ, the sizes of ints and enums and whatnot
may differ, the endianness of numbers may differ, and so on).
To reconcile their idiosyncratic internal representations ,
the two machines must agree on a common external representation
or "wire format." The sender converts its internal data to
the agreed-upon wire format and transmits it, and the receiver
converts the incoming bytes to its own internal representation.
Since the two internal representations may use different
byte counts to represent "the same" data, the number of bytes
used by one machine is pretty much irrelevant to the other,
and pretty much useless as a component of the wire format.
You might be interested in the number of bytes in the external
representation of some data -- for example, if your protocol
involves a header that declares how many payload bytes follow
it -- but that's not the same thing as the number of bytes
the local in-memory representation requires.
-- Er*********@sun .com
Eric Sosman wrote:
Alfonso Morra wrote On 09/28/05 17:21,:
ajm wrote:
Hi Alfonso,
1) no you do not need to disect your struct and compute the sum of the
component sizes in this fashion ! you can just apply sizeof() to
MyStruct and in fact it is better to do so since sizeof() includes any
padding etc. necessary to accommodate the structure components. Also I
see a strlen in there (?) your structure does not contain the strings
but merely pointers to them, it is the cost of these pointers that
contribute s to the size of the structure rather than the length of the
strings.
Think about it. If one stucture contains a string (char*) of length
1024, and the another variable of the same data type contains a string
(char*) of length 16, sizeof (DataStructure) will not report any
difference in size between the two variables. This is patently wrong
(see below)
I can't find anything called DataStructure in the
code you posted (see up-thread). Are you referring to
MyStruct? That seems likely, because the first question
you asked was
1). Is my calculation of the number of bytes used by
MyStruct correct?
For the usual C meaning of "number of bytes used" the
answer is "No." The size of MyStruct is sizeof(MyStruct ),
not the sum of the sizes of its elements plus the size of
other things in other places that some of its elements
happen to point at.
What you're really after, it seems, is not the size of
a MyStruct instance, but the number of bytes in a "serialized
version," that is, the length of some kind of encoding of
the data represented by a MyStruct and its dependents. This
latter count depends on the "on the wire" representation you
choose; if you like, it depends on your protocol.
2) you need to be clear by what you mean by "copy" a structure e.g., do
you want to "copy pointers" (after which both structures contains
pointers to the same string in memory) or do you want to "make copies
of pointers" (after which each structure has pointers to its own copies
of strings which could change over time) - in the former a memcpy of
size sizeof(MyStruct ) suffices, in the latter a component copy (perhaps
using strdup for char pointers) might be what you need.
I may have not asked the question correctly. But put simply, I need to
make a copy of a nested structure. A 'proper' copy, so that the
structure can be sent to another process on a different machine - you
may recall that I DID say that I was writing a messaging library. It
would be the second type of copying that I am interested in since the
data is being sent accross process boundaries and any raw pointers would
be meaningless in another process space.
Put simply, this is not what is usually called a "copy."
(Nor do I see anything in your code that I'd call a "nested
structure:" There's only one struct type shown, and it contains
no other struct types.)
You'll probably find it helpful to think not of "copies"
but of internal and external representations of your data.
The two machines may use entirely different representations
even if the MyStruct declarations are identical (the struct
padding may differ, the sizes of ints and enums and whatnot
may differ, the endianness of numbers may differ, and so on).
To reconcile their idiosyncratic internal representations ,
the two machines must agree on a common external representation
or "wire format." The sender converts its internal data to
the agreed-upon wire format and transmits it, and the receiver
converts the incoming bytes to its own internal representation.
Since the two internal representations may use different
byte counts to represent "the same" data, the number of bytes
used by one machine is pretty much irrelevant to the other,
and pretty much useless as a component of the wire format.
You might be interested in the number of bytes in the external
representation of some data -- for example, if your protocol
involves a header that declares how many payload bytes follow
it -- but that's not the same thing as the number of bytes
the local in-memory representation requires.
Hmmm, ok at least were on the right track here. Thanks for pointing out
the fact that the term "copying" here may have been "overloaded " and
thus potentially confusing.
What I mean copying means making a "complete and distinct" copy of a
memory block used by a data structure variable. the term 'DataStructure'
used above is just a place holder for an 'abstract data type'. Finally,
'nested' in this case refers to the fact that I have several pointers
(char*) and a union, "nested" within the MyStruct struct.
Regarding your low-level "close to the wire" concerns, I am using a
library which abstracts away, the low level stuff like endian
differences between machines etc. It provides a simple API like this
CreateMessage(D ataStructureToC opy, SizeofBinaryDat aBlock, MessageStructur e);
that allows creation of the message packet. I am shielded from the low
level stuff you mentioned earlier. So my question still remains:
Given a struct like MyStruct defined which contains "nested" members
like a char*, a union etc,
1). How can I calculate the size of the binary data block (to be used
internally by the function above to a call to memmove() )? - is my
calculation in my OP correct?
MTIA
Alfonso Morra <sw***********@ the-ring.com> writes:
[...] Hmmm, ok at least were on the right track here. Thanks for pointing
out the fact that the term "copying" here may have been "overloaded "
and thus potentially confusing.
What I mean copying means making a "complete and distinct" copy of a
memory block used by a data structure variable. the term
'DataStructure' used above is just a place holder for an 'abstract
data type'. Finally, 'nested' in this case refers to the fact that I
have several pointers (char*) and a union, "nested" within the
MyStruct struct.
The common term for this is a "deep copy".
--
Keith Thompson (The_Other_Keit h) ks***@mib.org <http://www.ghoti.net/~kst>
San Diego Supercomputer Center <*> <http://users.sdsc.edu/~kst>
We must do something. This is something. Therefore, we must do this.
Keith Thompson wrote: Alfonso Morra <sw***********@ the-ring.com> writes:
[...]
Hmmm, ok at least were on the right track here. Thanks for pointing
out the fact that the term "copying" here may have been "overloaded "
and thus potentially confusing.
What I mean copying means making a "complete and distinct" copy of a
memory block used by a data structure variable. the term
'DataStructur e' used above is just a place holder for an 'abstract
data type'. Finally, 'nested' in this case refers to the fact that I
have several pointers (char*) and a union, "nested" within the
MyStruct struct.
The common term for this is a "deep copy".
Tks for the clarification of the term - but do you know how I can
actually implement it in code?
Alfonso Morra wrote:
<snip> Hmmm, ok at least were on the right track here. Thanks for pointing out
the fact that the term "copying" here may have been "overloaded " and
thus potentially confusing.
What I mean copying means making a "complete and distinct" copy of a
memory block used by a data structure variable. the term 'DataStructure'
used above is just a place holder for an 'abstract data type'. Finally,
'nested' in this case refers to the fact that I have several pointers
(char*) and a union, "nested" within the MyStruct struct.
that is a "deep copy"
Regarding your low-level "close to the wire" concerns, I am using a
library which abstracts away, the low level stuff like endian
differences between machines etc. It provides a simple API like this
CreateMessage(D ataStructureToC opy, SizeofBinaryDat aBlock,
MessageStructur e);
that allows creation of the message packet. I am shielded from the low
level stuff you mentioned earlier. So my question still remains:
Given a struct like MyStruct defined which contains "nested" members
like a char*, a union etc,
1). How can I calculate the size of the binary data block (to be used
internally by the function above to a call to memmove() )? - is my
calculation in my OP correct?
The problem is that we don't know how CreateMessage works in detail so I
think you would get better answers from the author of the library unless
you can provide us with the full source for the CreateMessage function.
Given
struct struct_name struct_variable ;
The size you would pass to memcpy to copy struct_variable is "sizeof
struct_variable ". This would correctly handle any unions or nested
structures, but if there are pointers to other memory blocks (e.g. a
pointer to a string") and you want a deep copy then you need to write
additional code that knows about such fields, copies what they point to
separately, and then fixes up the pointers in the new structure. If the
"CreateMess age" function does this, then that's all well and good, but I
would be very surprised since your "MessageStructu re" parameter would
probably be a pointer to a tree (after all, you could pass the root node
of a tree to CreateMessage) which you would have spend significant
effort to construct. Then there is the question of how it is handled on
the wire which *will* affect the SizeOfBinaryDat aBlock if it can handle
the sort of thing you are trying.
My *guess* is that if you need to send a complex structure such as that
you showed earlier you will have to do it as several messages.
--
Flash Gordon
Living in interesting times.
Although my email address says spam, it is real and I read it.
Alfonso Morra wrote: [...]
1). How can I calculate the size of the binary data block (to be used
internally by the function above to a call to memmove() )? - is my
calculation in my OP correct?
The calculation in your O.P. produces nothing that seems
useful in connection with memmove().
As for the rest, while I recognize the words you are using,
it appears you are using them in ways unfamiliar to me. I sort
of thought I knew what you were trying to do, but the more you
write and the more I read the less I understand. Sorry.
--
Eric Sosman es*****@acm-dot-org.invalid
On Thu, 29 Sep 2005 00:35:26 +0000, Alfonso Morra wrote:
.... Hmmm, ok at least were on the right track here. Thanks for pointing out
the fact that the term "copying" here may have been "overloaded " and
thus potentially confusing.
What I mean copying means making a "complete and distinct" copy of a
memory block used by a data structure variable. the term 'DataStructure'
used above is just a place holder for an 'abstract data type'. Finally,
'nested' in this case refers to the fact that I have several pointers
(char*) and a union, "nested" within the MyStruct struct.
You need code that "knows about" the layut of your datastructure and can
make copies of all of the relevant parts.
Regarding your low-level "close to the wire" concerns, I am using a
library which abstracts away, the low level stuff like endian
differences between machines etc. It provides a simple API like this
CreateMessage(D ataStructureToC opy, SizeofBinaryDat aBlock, MessageStructur e);
The answers to your questions depend on how this function works. So the
only way to get answers to your questions is by reading the documentation.
However for it to be able to deal with representation issues like byte
order it must know about the layout of your structure.
that allows creation of the message packet. I am shielded from the low
level stuff you mentioned earlier. So my question still remains:
Given a struct like MyStruct defined which contains "nested" members
like a char*, a union etc,
1). How can I calculate the size of the binary data block (to be used
internally by the function above to a call to memmove() )? - is my
calculation in my OP correct?
That depends on what representation the function above uses to store the
serialised data. It must document a means to determine how much memory
is needed.
Lawrence This thread has been closed and replies have been disabled. Please start a new discussion. Similar topics | |
by: Bhalchandra Thatte |
last post by:
I am allocating a block of memory using malloc.
I want to use it to store a "header" structure
followed by structs in my application. How to calculate
the alignment without making any assumption
about the most restrictive type on my machine?
Thanks.
|
by: Kieran Simkin |
last post by:
Hi,
I wonder if anyone can help me, I've been headscratching for a few hours
over this.
Basically, I've defined a struct called cache_object:
struct cache_object {
char hostname;
char ipaddr;
};
|
by: phoenix |
last post by:
Hello,
I've got a design question. I need to keep track of some variables and I am
planning to put them inside a class or struct. Basically I'm talking about
10 bools, 20 ints and 2 arrays of ints. The size of the arrays would depend
on some external value (going from 0 to around 1000 max). I would have an
array of max 255 of these classes/structs (in most cases it will be less
then 5 however)
Since there's no real business logic my...
|
by: Kevin |
last post by:
Hi al
I have an interesting question.... I am working witha Win API this is the Function
Public Declare Function EnumJobs Lib "winspool.drv" Alias "EnumJobsA" (ByVal hPrinter As Long, ByVal FirstJob As Long, ByVal NoJobs As Long, ByVal Level As Long, pJob As Byte, ByVal cdBuf As Long, pcbNeeded As Long, pcReturned As Long) As Lon
Which I got from the API viewer that comes with VB 6. I have tried to convert it to the following
...
|
by: MSDousti |
last post by:
Hello
I want to write a C# (or VB.NET) program which reads the structure of
a PE (odinary win32 executable) file. These files have a long header
(512 bytes or so). The definition of the header exists in WINNT.h file
(you can see many "struct"s there).
Here comes my problem: how can I read a file and "fill" the structs?
| | |
by: Brian |
last post by:
I am "learning" C# and have run into a problem that, though I can work around
it, I would like to know what the *right* way to handle the issue is.
I have created an "Info" struct and assigned certain default values to it.
I have assigned this struct to a TreeView node via the Tag field. In my
"AfterSelect" processing call, I am casting the node tag value to my "Info"
struct. When I look at the fields they are set to the default values...
|
by: Bonj |
last post by:
I almost understand TSTs, to the point where I just need to know the answer
to this:
When making a TST (in C++) that will have as its leaf nodes words that make
up SQL language and an categorising identifier for each one, and each layer
of the tree will represent comparison of a further letter within the search
string, what will happen when a particular node is a leaf node itself, but
also has leaf nodes of its own? i.e. specifically, as...
|
by: heddy |
last post by:
I have an array of objects. When I use Array.Resize<T>(ref Object,int
Newsize); and the newsize is smaller then what the array was
previously, are the resources allocated to the objects that are now
thown out of the array released properly by the CLI?
|
by: ravi.sathyam |
last post by:
Hey,
So say I have a sockaddr_in struct stored in a packet which I receive
from my udp socket.....and it is stored within a certain offset into
this packet (which is basically a char array). Currently, the first
four bytes store an ID information and the next sizeof(struct
sockaddr_in) bytes store this struct.
Now, say that i declare a struct sockaddr_in temp_addr variable...would
the following lines be valid??
|
by: marktang |
last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look !
Part I. Meaning of...
|
by: Hystou |
last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it.
First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
| | |
by: Oralloy |
last post by:
Hello folks,
I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>".
The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed.
This is as boiled down as I can make it.
Here is my compilation command:
g++-12 -std=c++20 -Wnarrowing bit_field.cpp
Here is the code in...
|
by: jinu1996 |
last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth.
The Art of Business Website Design
Your website is...
|
by: Hystou |
last post by:
Overview:
Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
|
by: tracyyun |
last post by:
Dear forum friends,
With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
|
by: conductexam |
last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one.
At the time of converting from word file to html my equations which are in the word document file was convert into image.
Globals.ThisAddIn.Application.ActiveDocument.Select();...
|
by: adsilva |
last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
| | |
by: muto222 |
last post by:
How can i add a mobile payment intergratation into php mysql website.
| |
