PHP 5, XSL transformations of big files

[Meglio]

Hi.

I'm going to make in auto some tasks that I'm doing manually every
day. I'm uploading big XML file (~20mb) and then I'm using XSLT schema
to transform it to MySQL queries file.

I have used 3rd tools to process XSLT and it takes few seconds to
process my big XML file.

But now I'm moving to PHP so I'm trying to use XSL PHP extension to
apply my XSL file. But... it takes minutes... I allowed 10 minutes
( set_time_limit(600) ) and it still crashes and says that time
excited.

So please somebody help me to find a way out of the impasse. I can't
move forward now because of slow XSL processing.

The code I'm using is:

*******
// Load the XML source
$xml = new DOMDocument;
$xml->load('extracted/marketplace_feed_v1.xml');

$xsl = new DOMDocument;
$xsl->load('cb_marketplace_feed_compact.xsl');

// Configure the transformer
$proc = new XSLTProcessor;
$proc->importStyleSheet($xsl); // attach the xsl rules

$sqlCommands = $proc->transformToXML($xml);
if ($sqlCommands == FALSE)
{
echo "XSLT transformation failed";
exit;
}

if (file_put_contents('cbfeed.sql', $sqlCommands) == FALSE);
{
echo 'Failed to save XSLT transformation result to the file
"cbfeed.sql"';
exit;
}
*******

You can upload files used in my script here:

http://megliosoft.org/meglio/cbfeed/...ce_feed_v1.xml
http://megliosoft.org/meglio/cbfeed/...ed_compact.xsl
Any information may help.

Thanks,
Anton

[Thiago Macedo]

try to use javascript xslt transformation:
http://www.w3schools.com/xsl/xsl_client.asp

probaly faster

Meglio wrote:

Hi.

I'm going to make in auto some tasks that I'm doing manually every
day. I'm uploading big XML file (~20mb) and then I'm using XSLT schema
to transform it to MySQL queries file.

I have used 3rd tools to process XSLT and it takes few seconds to
process my big XML file.

But now I'm moving to PHP so I'm trying to use XSL PHP extension to
apply my XSL file. But... it takes minutes... I allowed 10 minutes
( set_time_limit(600) ) and it still crashes and says that time
excited.

So please somebody help me to find a way out of the impasse. I can't
move forward now because of slow XSL processing.

The code I'm using is:

*******
// Load the XML source
$xml = new DOMDocument;
$xml->load('extracted/marketplace_feed_v1.xml');

$xsl = new DOMDocument;
$xsl->load('cb_marketplace_feed_compact.xsl');

// Configure the transformer
$proc = new XSLTProcessor;
$proc->importStyleSheet($xsl); // attach the xsl rules

$sqlCommands = $proc->transformToXML($xml);
if ($sqlCommands == FALSE)
{
echo "XSLT transformation failed";
exit;
}

if (file_put_contents('cbfeed.sql', $sqlCommands) == FALSE);
{
echo 'Failed to save XSLT transformation result to the file
"cbfeed.sql"';
exit;
}
*******

You can upload files used in my script here:

http://megliosoft.org/meglio/cbfeed/...ce_feed_v1.xml
http://megliosoft.org/meglio/cbfeed/...ed_compact.xsl
Any information may help.

Thanks,
Anton

[Jerry Stuckle]

Thiago Macedo wrote:

try to use javascript xslt transformation:
http://www.w3schools.com/xsl/xsl_client.asp

probaly faster

Meglio wrote:

>Hi.

I'm going to make in auto some tasks that I'm doing manually every
day. I'm uploading big XML file (~20mb) and then I'm using XSLT schema
to transform it to MySQL queries file.

I have used 3rd tools to process XSLT and it takes few seconds to
process my big XML file.

But now I'm moving to PHP so I'm trying to use XSL PHP extension to
apply my XSL file. But... it takes minutes... I allowed 10 minutes
( set_time_limit(600) ) and it still crashes and says that time
excited.

So please somebody help me to find a way out of the impasse. I can't
move forward now because of slow XSL processing.

The code I'm using is:

*******
// Load the XML source
$xml = new DOMDocument;
$xml->load('extracted/marketplace_feed_v1.xml');

$xsl = new DOMDocument;
$xsl->load('cb_marketplace_feed_compact.xsl');

// Configure the transformer
$proc = new XSLTProcessor;
$proc->importStyleSheet($xsl); // attach the xsl rules

$sqlCommands = $proc->transformToXML($xml);
if ($sqlCommands == FALSE)
{
echo "XSLT transformation failed";
exit;
}

if (file_put_contents('cbfeed.sql', $sqlCommands) == FALSE);
{
echo 'Failed to save XSLT transformation result to the file
"cbfeed.sql"';
exit;
}
*******

You can upload files used in my script here:

http://megliosoft.org/meglio/cbfeed/...ce_feed_v1.xml
http://megliosoft.org/meglio/cbfeed/...ed_compact.xsl
Any information may help.

Thanks,
Anton

How is that going to help? Javascript is client side - he'd have to
download those large files, then do it on the client and finally upload
the results. And clients machines are typically less powerful than servers.

A very bad suggestion, IMHO.

--
==================
Remove the "x" from my email address
Jerry Stuckle
JDS Computer Training Corp.
js*******@attglobal.net
==================

[Thiago Macedo]

On May 3, 1:05 am, Jerry Stuckle <[e-mail]:

Thiago Macedo wrote:

try to use javascript xslt transformation:
http://www.w3schools.com/xsl/xsl_client.asp

probaly faster

Meglio wrote:

Hi.

I'm going to make in auto some tasks that I'm doing manually every
day. I'm uploading big XML file (~20mb) and then I'm using XSLT schema
to transform it to MySQL queries file.

I have used 3rd tools to process XSLT and it takes few seconds to
process my big XML file.

But now I'm moving to PHP so I'm trying to use XSL PHP extension to
apply my XSL file. But... it takes minutes... I allowed 10 minutes
( set_time_limit(600) ) and it still crashes and says that time
excited.

So please somebody help me to find a way out of the impasse. I can't
move forward now because of slow XSL processing.

The code I'm using is:

*******
// Load the XML source
$xml = new DOMDocument;
$xml->load('extracted/marketplace_feed_v1.xml');

$xsl = new DOMDocument;
$xsl->load('cb_marketplace_feed_compact.xsl');

// Configure the transformer
$proc = new XSLTProcessor;
$proc->importStyleSheet($xsl); // attach the xsl rules

$sqlCommands = $proc->transformToXML($xml);
if ($sqlCommands == FALSE)
{
echo "XSLT transformation failed";
exit;
}

if (file_put_contents('cbfeed.sql', $sqlCommands) == FALSE);
{
echo 'Failed to save XSLT transformation result to the file
"cbfeed.sql"';
exit;
}
*******

You can upload files used in my script here:

>http://megliosoft.org/meglio/cbfeed/...ce_feed_v1.xml
http://megliosoft.org/meglio/cbfeed/...ed_compact.xsl

Any information may help.

Thanks,
Anton

How is that going to help? Javascript is client side - he'd have to
download those large files, then do it on the client and finally upload
the results. And clients machines are typically less powerful than servers.

A very bad suggestion, IMHO.

--
==================
Remove the "x" from my email address
Jerry Stuckle
JDS Computer Training Corp.
[e-mail]
==================

sorry for not acquiring your "requisites".. but, if it will run on a
server, instead of download this large file, he'll upload it.. and
this is usually worst.
by the way, probaly the timeout is occurring in the file upload, not
in the transformation. already examined where it stops the execution?

regards,

Thiago

[=?ISO-8859-15?Q?Iv=E1n_S=E1nchez_Ortega?=]

Jerry Stuckle wrote:

Thiago Macedo wrote:

>try to use javascript xslt transformation:

[...]

How is that going to help?

By not crashing. This one's a quick & dirty automation task, so it ain't to
be pretty.

Javascript is client side - he'd have to download those large files, then
do it on the client and finally upload the results. And clients machines
are typically less powerful than servers.

We can suppose the O.P. is inside a LAN and that semi-large files are not a
problem.


Meglio wrote:

I have used 3rd tools to process XSLT and it takes few seconds to
process my big XML file.

But now I'm moving to PHP so I'm trying to use XSL PHP extension to
apply my XSL file. But... it takes minutes...

I might get kicked out of here for saying here, but maybe PHP isn't the
right tool for this particular job.

You can consider using PHP as a web frontend, and use exec() to call a
command-line xsltproc, or any other 3rd-party tool. You can also do
client-side processing, given you have set the size limits for uploads
right.

I allowed 10 minutes ( set_time_limit(600) ) and it still crashes and says
that time excited.

Can we have a look at the *exact* error message?
--
----------------------------------
Iván Sánchez Ortega -ivansanchez-algarroba-escomposlinux-punto-org-

Y dijo el capitán Bloub: "Abordar el barco"... y les quedó precioso.

[Meglio]

On May 3, 7:15 am, Thiago Macedo <[e-mail]:

On May 3, 1:05 am, Jerry Stuckle <[e-mail]:

Thiago Macedo wrote:

try to use javascript xslt transformation:
>http://www.w3schools.com/xsl/xsl_client.asp

probaly faster

Meglio wrote:
>Hi.

>I'm going to make in auto some tasks that I'm doing manually every
>day. I'm uploading big XML file (~20mb) and then I'm using XSLT schema
>to transform it to MySQL queries file.

>I have used 3rd tools to process XSLT and it takes few seconds to
>process my big XML file.

>But now I'm moving to PHP so I'm trying to use XSL PHP extension to
>apply my XSL file. But... it takes minutes... I allowed 10 minutes
>( set_time_limit(600) ) and it still crashes and says that time
>excited.

>So please somebody help me to find a way out of the impasse. I can't
>move forward now because of slow XSL processing.

>The code I'm using is:

>*******
>// Load the XML source
>$xml = new DOMDocument;
>$xml->load('extracted/marketplace_feed_v1.xml');

>$xsl = new DOMDocument;
>$xsl->load('cb_marketplace_feed_compact.xsl');

>// Configure the transformer
>$proc = new XSLTProcessor;
>$proc->importStyleSheet($xsl); // attach the xsl rules

>$sqlCommands = $proc->transformToXML($xml);
>if ($sqlCommands == FALSE)
>{
> echo "XSLT transformation failed";
> exit;
>}

>if (file_put_contents('cbfeed.sql', $sqlCommands) == FALSE);
>{
> echo 'Failed to save XSLT transformation result to the file
>"cbfeed.sql"';
> exit;
>}
>*******

>You can upload files used in my script here:

>>http://megliosoft.org/meglio/cbfeed/...ce_feed_v1.xml
>>http://megliosoft.org/meglio/cbfeed/...ed_compact.xsl

>Any information may help.

>Thanks,
>Anton

How is that going to help? Javascript is client side - he'd have to
download those large files, then do it on the client and finally upload
the results. And clients machines are typically less powerful than servers.

A very bad suggestion, IMHO.

--
==================
Remove the "x" from my email address
Jerry Stuckle
JDS Computer Training Corp.
[e-mail]
==================

sorry for not acquiring your "requisites".. but, if it will run on a
server, instead of download this large file, he'll upload it.. and
this is usually worst.
by the way, probaly the timeout is occurring in the file upload, not
in the transformation. already examined where it stops the execution?

regards,

Thiago

>if it will run on a server, instead of download this large file, he'll upload it.. and

this is usually worst.

THIS is not worst because when I do it manually I still transform XML
to SQL (which is big file too) and upload it to the server. So I have
to upload SQL to the server in any case. The goal is to upload XML
directly to the server on auto and to transform it to SQL file using
PHP script.

>by the way, probably the timeout is occurring in the file upload, not

in the transformation. already examined where it stops the execution?

I'm not so stupid to ask question before to examine where it stops. It
freezes on XSL transformation.

[Meglio]

Fatal error: Maximum execution time of ... seconds exceeded in ... on
line ...

[Piotr]

Meglio wrote:

Hi.

I'm going to make in auto some tasks that I'm doing manually every
day. I'm uploading big XML file (~20mb) and then I'm using XSLT schema
to transform it to MySQL queries file.

I have used 3rd tools to process XSLT and it takes few seconds to
process my big XML file.

But now I'm moving to PHP so I'm trying to use XSL PHP extension to
apply my XSL file. But... it takes minutes... I allowed 10 minutes
( set_time_limit(600) ) and it still crashes and says that time
excited.

So please somebody help me to find a way out of the impasse. I can't
move forward now because of slow XSL processing.

Maybe there is some issue that causes PHP xslt processor to fail or go
into infinite loop. Try cutting the xsl template to few smaller parts
(just for test) and see if the error occurs in any specyfic part. Maybe
there is some syntax used that PHP doesn't really like..

Not an xsl expert, so if this is way off, just ignore..

best regards
Piotr N

[Meglio]

On May 3, 11:29 am, Piotr <[e-mail]:

Meglio wrote:

Hi.

I'm going to make in auto some tasks that I'm doing manually every
day. I'm uploading big XML file (~20mb) and then I'm using XSLT schema
to transform it to MySQL queries file.

I have used 3rd tools to process XSLT and it takes few seconds to
process my big XML file.

But now I'm moving to PHP so I'm trying to use XSL PHP extension to
apply my XSL file. But... it takes minutes... I allowed 10 minutes
( set_time_limit(600) ) and it still crashes and says that time
excited.

So please somebody help me to find a way out of the impasse. I can't
move forward now because of slow XSL processing.

Maybe there is some issue that causes PHP xslt processor to fail or go
into infinite loop. Try cutting the xsl template to few smaller parts
(just for test) and see if the error occurs in any specyfic part. Maybe
there is some syntax used that PHP doesn't really like..

Not an xsl expert, so if this is way off, just ignore..

best regards
Piotr N

Hi. Thanks for you advice. But i'm not XSLT expert too so it is not
simple way for me to find such bug. So I'm still looking for an expert
who can help me to understand where the problems is.

Thanks,
Anton

[Piotr]

Meglio wrote:

>Maybe there is some issue that causes PHP xslt processor to fail or go
into infinite loop. Try cutting the xsl template to few smaller parts
(just for test) and see if the error occurs in any specyfic part. Maybe
there is some syntax used that PHP doesn't really like..

Not an xsl expert, so if this is way off, just ignore..

best regards
Piotr N

Hi. Thanks for you advice. But i'm not XSLT expert too so it is not
simple way for me to find such bug. So I'm still looking for an expert
who can help me to understand where the problems is.

Thanks,
Anton

Afaik, you should find tags like his:

<xsl:template match="some_tag">
[..]
</xsl:template>

if you have more then one, simply comment all but one, see if this times
out, if it doesn't, comment this template out and uncomment next one. Do
so as long as you find the one that times out.

When you find that one, comment out some functions, like xsl:for-each.
To comment, use <!-- --syntax.

I suppose there is also xsl group for xslt specific problems.

best regards
Piotr N

[Meglio]

On May 3, 12:20 pm, Piotr <[e-mail]:

Meglio wrote:

Maybe there is some issue that causes PHP xslt processor to fail or go
into infinite loop. Try cutting the xsl template to few smaller parts
(just for test) and see if the error occurs in any specyfic part. Maybe
there is some syntax used that PHP doesn't really like..

Not an xsl expert, so if this is way off, just ignore..

best regards
Piotr N

Hi. Thanks for you advice. But i'm not XSLT expert too so it is not
simple way for me to find such bug. So I'm still looking for an expert
who can help me to understand where the problems is.

Thanks,
Anton

Afaik, you should find tags like his:

<xsl:template match="some_tag">
[..]
</xsl:template>

if you have more then one, simply comment all but one, see if this times
out, if it doesn't, comment this template out and uncomment next one. Do
so as long as you find the one that times out.

When you find that one, comment out some functions, like xsl:for-each.
To comment, use <!-- --syntax.

I suppose there is also xsl group for xslt specific problems.

best regards
Piotr N

I'm not sure I'll get working XSL template if I comment some
instructions.

[Piotr]

Meglio wrote:

>
I'm not sure I'll get working XSL template if I comment some
instructions.

It wont produce any good result for sure, but at this point, you
are just checking for what is causing the problem with PHP.
But as i said before, maybe it would be better to ask on xsl group about
template debugging.

best regards
Piotr N

[Meglio]

On May 3, 12:20 pm, Piotr <[e-mail]:

Meglio wrote:

Maybe there is some issue that causes PHP xslt processor to fail or go
into infinite loop. Try cutting the xsl template to few smaller parts
(just for test) and see if the error occurs in any specyfic part. Maybe
there is some syntax used that PHP doesn't really like..

Not an xsl expert, so if this is way off, just ignore..

best regards
Piotr N

Hi. Thanks for you advice. But i'm not XSLT expert too so it is not
simple way for me to find such bug. So I'm still looking for an expert
who can help me to understand where the problems is.

Thanks,
Anton

Afaik, you should find tags like his:

<xsl:template match="some_tag">
[..]
</xsl:template>

if you have more then one, simply comment all but one, see if this times
out, if it doesn't, comment this template out and uncomment next one. Do
so as long as you find the one that times out.

When you find that one, comment out some functions, like xsl:for-each.
To comment, use <!-- --syntax.

I suppose there is also xsl group for xslt specific problems.

best regards
Piotr N

I'm not sure whether this is PHP or XSLT problem so I have created
another thread in XSLT group too:
http://groups.google.com/group/micro...fd7cc536afaa3#