Parsing a generic data file

[Jasper]

Hi, Maybe this is off-topic, but perhaps you can help. I'm looking for ideas
on how to parse a data file.

I dont know XML but I know it parses data in text format.

I have a structured data file of the general form shown below. I dont have
any definition of the data. Basically it looks like it is hierarchical,
token/data pairs defined by brackets and square brackets.

I would like to parse this out into some sort of data object(s) in C++ so
that I can gain programmatic access to the variables.

My app is C++ so the solution must be the same. Also it must be very
lightweight and *very* fast as I must decode multiple pages in realtime.

Would adapting an XML parser to do this be a possible solution?

Any pointers/ideas/references/code snippets/observations appreciated.

TIA

Basic example showing data structure (whitespaces and carriage returns added
by me for clarity).

{

"teacher":{
"name":
"Mr Borat",
"age":
"35",
"Nationality":
"Kazakhstan"},
"Class":{
"Semester":
"Summer",
"Room":
null,
"Subject":
"Politics",
"Notes":
"We're happy, you happy?"},

"Students":
[
{
"Smith":
[{"First Name":"Mary","sex":"Female"}],
"Brown":
[{"First Name":"John","sex":"Male"}],
"Jackson":
[{"First Name":"Jackie","sex":"Female"}]
}
],
"Grades":
[
{
"Test":
[{"grade":"A","points":68},{"grade":"B","points":25 },{"grade":"C","points":15}],
"Test":
[{"grade":"C","points":2},{"grade":"B","points":29} ,{"grade":"A","points":55}],
"Test":
[{"grade":"C","points":2},{"grade":"A","points":72} ,{"grade":"A","points":65}]
}
]

}

[msnews.microsoft.com]

"Jasper" <no*******@dontmail.comwrote in message
news:2p*********************@pipex.net...

Hi, Maybe this is off-topic, but perhaps you can help. I'm looking for
ideas on how to parse a data file.

I dont know XML but I know it parses data in text format.

I have a structured data file of the general form shown below. I dont have
any definition of the data. Basically it looks like it is hierarchical,
token/data pairs defined by brackets and square brackets.

I would like to parse this out into some sort of data object(s) in C++ so
that I can gain programmatic access to the variables.

My app is C++ so the solution must be the same. Also it must be very
lightweight and *very* fast as I must decode multiple pages in realtime.

Would adapting an XML parser to do this be a possible solution?

Any pointers/ideas/references/code snippets/observations appreciated.

TIA

Basic example showing data structure (whitespaces and carriage returns
added by me for clarity).

{

"teacher":{
"name":
"Mr Borat",
"age":
"35",
"Nationality":
"Kazakhstan"},
"Class":{
"Semester":
"Summer",
"Room":
null,
"Subject":
"Politics",
"Notes":
"We're happy, you happy?"},

"Students":
[
{
"Smith":
[{"First Name":"Mary","sex":"Female"}],
"Brown":
[{"First Name":"John","sex":"Male"}],
"Jackson":
[{"First Name":"Jackie","sex":"Female"}]
}
],
"Grades":
[
{
"Test":

[{"grade":"A","points":68},{"grade":"B","points":25 },{"grade":"C","points":15}],
"Test":

[{"grade":"C","points":2},{"grade":"B","points":29} ,{"grade":"A","points":55}],
"Test":

[{"grade":"C","points":2},{"grade":"A","points":72} ,{"grade":"A","points":65}]
}
]

}

Looks like JSON to me, search for a JSON library.
JSON is a way of representing objects using string literals that is used for
passing information to clients that use JavaScript.

--

Joe Fawcett (MVP - XML)
http://joe.fawcett.name

[Jasper]

"msnews.microsoft.com" <jo********@newsgroup.nospamwrote in message
news:uC**************@TK2MSFTNGP03.phx.gbl...

"Jasper" <no*******@dontmail.comwrote in message
news:2p*********************@pipex.net...

>Hi, Maybe this is off-topic, but perhaps you can help. I'm looking for
ideas on how to parse a data file.

Looks like JSON to me, search for a JSON library.
JSON is a way of representing objects using string literals that is used
for passing information to clients that use JavaScript.

Does it? Makes sense if that's true. I was sure it fit some sort of "web
format" but I didn't know which.
I presume there must be some sort of C++ code available to parse it out.

I'll take a look.

Thanks

[Lynn]

"Jasper" <no*******@dontmail.comwrote in message
news:2p*********************@pipex.net...

Hi, Maybe this is off-topic, but perhaps you can help. I'm looking for
ideas on how to parse a data file.

can't you create arrays of C++ structs or classes to hold this data? As for
parsing it, if you don't want to write your own parser there has to be an
abundance of libraries out there you could use out of the box, no?
Efficiency will vary but I can't see why any decent commercial product, if
not your own code, would not be *very* fast

I guess I'm not seeing why you would use XML or XML tools to intermediate
this process when the data is not coming at you in XML and you've given no
indication that you need to out it as XML for other processes to consume ...
?

I dont know XML but I know it parses data in text format.

I have a structured data file of the general form shown below. I dont have
any definition of the data. Basically it looks like it is hierarchical,
token/data pairs defined by brackets and square brackets.

I would like to parse this out into some sort of data object(s) in C++ so
that I can gain programmatic access to the variables.

My app is C++ so the solution must be the same. Also it must be very
lightweight and *very* fast as I must decode multiple pages in realtime.

Would adapting an XML parser to do this be a possible solution?

Any pointers/ideas/references/code snippets/observations appreciated.

TIA

Basic example showing data structure (whitespaces and carriage returns
added by me for clarity).

{

"teacher":{
"name":
"Mr Borat",
"age":
"35",
"Nationality":
"Kazakhstan"},
"Class":{
"Semester":
"Summer",
"Room":
null,
"Subject":
"Politics",
"Notes":
"We're happy, you happy?"},

"Students":
[
{
"Smith":
[{"First Name":"Mary","sex":"Female"}],
"Brown":
[{"First Name":"John","sex":"Male"}],
"Jackson":
[{"First Name":"Jackie","sex":"Female"}]
}
],
"Grades":
[
{
"Test":

[{"grade":"A","points":68},{"grade":"B","points":25 },{"grade":"C","points":15}],
"Test":

[{"grade":"C","points":2},{"grade":"B","points":29} ,{"grade":"A","points":55}],
"Test":

[{"grade":"C","points":2},{"grade":"A","points":72} ,{"grade":"A","points":65}]
}
]

}

[Anthony Jones]

"Jasper" <no*******@dontmail.comwrote in message
news:Or******************************@pipex.net...

>
"msnews.microsoft.com" <jo********@newsgroup.nospamwrote in message
news:uC**************@TK2MSFTNGP03.phx.gbl...

"Jasper" <no*******@dontmail.comwrote in message
news:2p*********************@pipex.net...

Hi, Maybe this is off-topic, but perhaps you can help. I'm looking for
ideas on how to parse a data file.

Looks like JSON to me, search for a JSON library.
JSON is a way of representing objects using string literals that is used
for passing information to clients that use JavaScript.

Does it? Makes sense if that's true. I was sure it fit some sort of "web
format" but I didn't know which.
I presume there must be some sort of C++ code available to parse it out.

It is JSON. You would need to be looking at the Javascript eval method to
parse it. The returned object would then have a heiarchy you could pull
data from e.g.:-

var x = o.Class.Subject

x == "Politics" // will be true

However the structure is somewhat suspect.

The students array contains only one object on which all students are
placed. Each student having their last name as the attribute ID for their
object (what happens if the class is attended by more than one Smith?).
This object is in turn an array containing only one object.

The Grades array suffers the same problem where again inappropriate use of
{ } causes the array to contain only one object and in this case the same
identifier "Test" used multiple times resulting in it being redefined and
only containing the last entry.

Here is a cleaner version (although I'm not entirely happy with the
identifiers "Last Name" and "First Name" containing a space it is legal):-

{

"teacher":{
"name": "Mr Borat",
"age": 35,
"Nationality": "Kazakhstan"
},
"Class":{
"Semester": "Summer",
"Room": null,
"Subject": "Politics",
"Notes": "We're happy, you happy?"
},

"Students":
[
{"Last Name":"Smith",
"First Name":"Mary","sex":"Female"},
{"Last Name":"Brown",
"First Name":"John","sex":"Male"},
{"Last Name":"Jackson",
"First Name":"Jackie","sex":"Female"}
],
"Grades":
[
{"Test":"Name of a Test",
Points: {"A":68,"B":25,"C":15}}
{"Test":"Name of a different test",
Points: {"A":55,"B":29,"C":2}}
{"Test": "Name of yet another test",
Points: {"A":72,"B":65,"C":2}}
]

}
--
Anthony Jones - MVP ASP/ASP.NET

[Anthony Jones]

<dn********@gmail.comwrote in message
news:da**********************************@e23g2000 prf.googlegroups.com...

The question arises as to whether the output XML should represent the

data

that would be available in the set of generated objects had the JSON

been

eval'd?

Perhaps the Grades section should look like this:-

<Grades>
<Test>
<grade>C</grade>
<points>2</points>
</Test>
<Test>
<grade>A</grade>
<points>72</points>
</Test>
<Test>
<grade>A</grade>
<points>65</points>
</Test>
</Grades>

since only this data would appear in the an eval of the JSON?

The answer is clearly: No.

Oh, I thought the raison d'être behind JSON was that a data structure could
be serialised to a string that could be passed to Javascript and
re-assembled easily by using the Eval statement.

It is the definition of JSON (and the convertors from XML to JSON use
this) that a sequence of repeating xml elements with the same name are
represented as an ARRAY in JSON.

Is there a spec? Where does it say that?

>
We don't care what an JScript interpreter would do with the data, but
we must implement a truthful and lossless conversion. Not producing
all <test /and <grade /elements results in data loss.

Agreed. I'm willing to be shown wrong on this but if you're right than JSON
is bust and pointless.

--
Anthony Jones - MVP ASP/ASP.NET

[Anthony Jones]

<dn********@gmail.comwrote in message
news:f2**********************************@a35g2000 prf.googlegroups.com...

I also think that a more appropriate JSON representation than:

"Grades":
[
{
"Test":
[{"grade":"A","points":68},{"grade":"B","points":25 },
{"grade":"C","points":15}],
"Test":
[{"grade":"C","points":2},{"grade":"B","points":29} ,
{"grade":"A","points":55}],
"Test":
[{"grade":"C","points":2},{"grade":"A","points":72} ,
{"grade":"A","points":65}]
}
]
should have been:

"Grades":

{
"Test":
[
{"grade":"A","points":68,"grade":"B","points":
25,"grade":"C","points":15},

{"grade":"C","points":2, "grade":"B","points":29,
"grade":"A","points":55},

{"grade":"C","points":2, "grade":"A","points":72,
"grade":"A","points":65}
]
}

We're just guessing at the intent but that appears to be an object called
Grades that contains just one member an array called Test containing what
appears to be grades required to pass each test. Seems a little convoluted
and how is each test identified? Ordinal position?

Also, instead of:

"Students":
[
{
"Smith":
[{"First Name":"Mary","sex":"Female"}],
"Brown":
[{"1First Name":"John","sex":"Male"}],
"Jackson":
[{"2First Name":"Jackie","sex":"Female"}]
}
],

it is better to have just:

"Students":
{
"Smith":
{"First Name":"Mary","sex":"Female"},
"Brown":
{"1First Name":"John","sex":"Male"},
"Jackson":
{"2First Name":"Jackie","sex":"Female"}
}
,

And if you have two students with the last name Smith? Smith magically
becomes an array?

>Maybe, the original data was produced by a faulty XML --JSON
convertor.

Its difficult to make sense of what appears to be faulty both as JSON and as
a logical structure.

--
Anthony Jones - MVP ASP/ASP.NET