Dynamic pages vs search engines

[Sandy Pittendrigh]

I have a how-to-do-it manual like site,
related to fishing. I want to add a new
interactive question/comment feature to each
instructional page on the site.

I want (registered) users to be able to add
comments at the bottom of each page, similar
to the way the php, mysql, apache manuals work.

PUNCHLINE_A:
I don't want to revert to a system of dynamic links
that look like "pagePainter.php?page_id=123"

I did that for a while. But every time I edited
the site (reloaded the database) all the page_ids
changed, and then voila I had stale links spattered
all over my Google search results--because the
same page now had a slightly different page_id.

To avoid that I could associate each page with
a unique lookup string that is its server-side file path:

pagePainter.php?page_key=/pages/manual/chapter1/engine_repair/carburetors

Then the keys wouldn't change everytime I added a few new pages
to the system, and reloaded the database. Then my Google search
links would remain constant.

PUNCHLINE_B:
Will the search engines spider and index links that look like that?
Or will I have to make a system that spits out the entire page as static
html, each time a registered user adds a new page question or comment?

Or should I put the comments box in an iframe, which is also a spidering
can of worms?

[Andy Jeffries]

On Tue, 04 Apr 2006 06:54:44 -0600, Sandy Pittendrigh wrote:

pagePainter.php?page_key=/pages/manual/chapter1/engine_repair/carburetors

PUNCHLINE_B:
Will the search engines spider and index links that look like that? Or
will I have to make a system that spits out the entire page as static
html, each time a registered user adds a new page question or comment?

You could take the best of both worlds and use something like:

pagePainter.php/pages/manual/chapter1/engine_repair/carburetors

The page_key data would be available in pagePainter.php as
$_SERVER["PATH_INFO"] providing you're running PHP as an apache module.

Cheers,
Andy

--
Andy Jeffries MBCS CITP ZCE | gPHPEdit Lead Developer
http://www.gphpedit.org | PHP editor for Gnome 2
http://www.andyjeffries.co.uk | Personal site and photos

[John Dunlop]

Sandy Pittendrigh:

[re a page that comments can be added to but that stays essentially the
same]

PUNCHLINE_A:
I don't want to revert to a system of dynamic links
that look like "pagePainter.php?page_id=123"
No, neither would I, unless I could explain exactly why each part of
the URL was there.
To avoid that I could associate each page with
a unique lookup string that is its server-side file path:
I think the idea of having persistent URLs is sound.
pagePainter.php?page_key=/pages/manual/chapter1/engine_repair/carburetors

Can you explain why each part of the URL is there? For example, why
is <pagePainter> there?

--
Jock

[rlee0001]

Sandy,

Why are you passing the page URI as a parameter? The capability you
request has nothing to do with the URL. For example, you could use...

http://www.example.com/pages/manual/...arburetors.php

And have an interactive comments section at the bottom of that page.
Also, you can use numbers as in your first example that are not
autonumbered by the database. Using static numbers gives you the
requirement that reloading the database doesn't re-assign record
numbers.

Is all your content sitting in a database now or is it static? Create a
function:

function docomments ($page) {
// OUTPUT NEW COMMENT FORM
// QUERY & SHOW COMMENTS
return null;
}

Where $page is the request URI. In the database, store each comment
with the "parent_page" varchar. This field just coresponds to the
request URI. Pretty simple stuff. The docomments function can be loaded
via a global include file for easy maintenance. A seperate page can
handle processing new incoming comments for the whole site. Just pass
in the $page value as a hidden form field to add the comment to the
database. Redirect users back to that URI.

-Robert

[Sandy.Pittendrigh]

>> pagePainter.php?page_key=/pages/manual/chapter1/engine_repair/carburetors

Can you explain why each part of the URL is there? For example, why

is <pagePainter> there?

If you want to dynamically generate the page, you need
a php script to generate the page? No? The name of this
script is pagePainter.php

page_key is a GET parameter
/pages/manual/chapter1/engine_repair/carburetors is the string
value of that get parameter. It can then be used in a mysql query:

select * from my_pages where page_key
='/pages/manual/chapter1/engine_repair/carburetors'

.....further, that key is guaranteed to be unique, because it
comes from a file path, and
more important, guaranteed to stay the same each
time the database is reloaded (by a program that
recursively walks the server-side file system, doing
magic stuff along the way)

[Sandy.Pittendrigh]

I think there is some confusion here about what I was proposing.
If you generate pages using the result of a mysql query, it is
tempting to use the numerical primary key of the page table
as the select criteria:

select * from my_pages where page_id=1
In order to make that query, the source page might
have a link like pagePainter.php?page_id=42
.....but that is a bad idea, because Google will cache that
link, and the relevant page_id changes each time the database
is reloaded.

If the database search key was made from a string,
where that string was the 'logical' server-side file path,
then it would still be just as unique as a primary key,
but it would not change each time the database was
reloaded.

However, and this was my question, sort of:
some search engines (is this true?) don't like to index
what look like dynamic links. Google does, I'm not
sure they all do. One way to guarantee indexing
is to produce static html (from a database) rather
than dynamic html.

Andy Jeffries (I think) showed me how to pass
a string GET parameter (a long path string) that
I could use as in mysql query (just like a page_id)
that would not look to search engines like a dynamic link.

[rlee0001]

> ...and the relevant page_id changes each time the database is reloaded.

Not if you don't use an autonumber field. A number is just as static as
a string. There are advantages of using a string but "staticness" is
not one of them.

Why are you using pagepainter.php? What does this script do? I'll ask
again, what is wrong with this:

/pages/manual/chapter1/engine_repair/carburetor.php

?

Another words, why are you passing the path as a parameter instead of
allowing the visitor to navigate to the URL directly? The URL being
passed as a parameter has nothing to do with putting an interactive
comments section at the bottom of the page.

Also, I wouldn't worry about search engines indexing a dynamic site.
Google does it and within the next 18 months they will go from 48% of
the search market to almost 80%. They are the only engine that really
matters anymore. Search engines that refuse to index dynamic pages are
just digging themselves a grave. Let them; they are better off dead.

-Robert

[Sandy Pittendrigh]

rlee0001 wrote:

...and the relevant page_id changes each time the database is reloaded.

Not if you don't use an autonumber field. A number is just as static as
a string. There are advantages of using a string but "staticness" is
not one of them.
Possible yes. But maybe not so easily.
I currently use a CMS that I wrote, that walks down through
a "source" directory tree, finding files (including special mime types
of my own invention) and then creates logical pages in a schema.
pagePainter.php gets passed a page_id and then recursively writes
out complete html pages (with consistent navigation, look and feel)
as static html, in a parallel directory tree to the soruce tree.

Works great. I can manage hundreds of pages with a little editing
and a mouse click. I could use a static, non-changing number as
the lookup key, but it would take some head-scratching to figure
out how to generate the same unchanging number for each page, upon each
database reload. It would be easy to concatenate a path string
from the source tree and use that as a unique lookup string.

But, my current 'generate static pages' system will not work
for pages that would have a built in forum or bulletin board
at the bottom of each page. So now I want to figure out
how to have dynamic pages whose lookup key doesn't change
for each database reload.

Why are you using pagepainter.php? What does this script do? I'll ask
again, what is wrong with this:
pagePainter.php exists at one location. It takes a lookup key
and they (actually either, depending on an optional parameter)
either generates dynamic or static html.

/pages/manual/chapter1/engine_repair/carburetor.php
carburetor.php is a script? I don't want to write a different
script for each of several hundred pages. I want one script
to dynamically generate several hundred pages from one script,
where each page differs from the next according to the information
that comes from a mysql query....a query that pagePainter.php
makes.
Another words, why are you passing the path as a parameter instead of
allowing the visitor to navigate to the URL directly? The URL being
passed as a parameter has nothing to do with putting an interactive
comments section at the bottom of the page.
the url doesn't exist until pagePainter.php creates it.

Also, I wouldn't worry about search engines indexing a dynamic site.

That would be good news.
That was the punchline question of my original post.
Search engines didn't always index dynamic pages. Maybe it's not
an issue anymore. I'm a programmer, but not a web developer
by trade. The web stuff is a hobby.....so I'm good at it,
but I don't always keep up with the latest gossip
and developments. There was a time when dynamic links
were a search engine problem and question.

[John Dunlop]

Sandy Pittendrigh:

If you want to dynamically generate the page, you need
a php script to generate the page? No? The name of this
script is pagePainter.php

Since users interact with URLs not with filenames and you, as a
server, interact with filenames after mapping them from URLs, there are
different pressures on what forms URLs take than there are on what
forms filenames take. There doesn't have to be a one-to-one mapping
between URL and filename. In other words, the URL doesn't have to
correspond in part or in whole to the name of the script.

mod_rewrite:

http://httpd.apache.org/docs/1.3/mod/mod_rewrite.html

--
Jock