473,326 Members | 2,095 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,326 software developers and data experts.

website to expire...how to get Spiders to redirect to other site?

RTL
Hello all,

Actually, we're letting several of our domain names go back into the field.
I believe there must be some industry way to use some HTML element/tag to
get the Spiders to start updating their links to our main site for the
remainder of the domain's life.

I have found this meta element/tag, but that doesn't help with what we want
to accomplish:

<meta name="revisit-after" content="5 days">

I've also gone to alt.spiders, but there were no messages.

Any help would be appreicated!

Rich
www.techarts.com
Jul 20 '05 #1
3 2588
RTL wrote:

we're letting several of our domain names go back into the field. I
believe there must be some industry way to use some HTML
element/tag to get the Spiders to start updating their links to our
main site for the remainder of the domain's life.
This OT for an html authoring group.

If each domain name has its own ip address, i.e., uris point to
different resources, then you should use server redirect to point
requests for about-to-expire uris to the domain you will keep. Make
sure to specify that the redirect is permanent. You didn't specify
the server software. If it's Apache, search the apache software site
for redirect.

When a robot encounters a permanent redirect, it will update its database.
I have found this meta element/tag, but that doesn't help with what
we want to accomplish:

<meta name="revisit-after" content="5 days">
meta tags are a poor replacement for real server headers. For real
server headers, you want redirect.
I've also gone to alt.spiders, but there were no messages.


alt.www.webmasters
comp.infosystems.www.servers.unix (if your server is unix-based)
alt.apache.configuration

But better than usenet groups, try the web. There are loads of how-to
resources for this sort of simple question. Always try the web before
usenet.

--
Brian
follow the directions in my address to email me

Jul 20 '05 #2
Tim
On Thu, 18 Sep 2003 13:45:15 GMT,
"RTL" <bl*******************@nospam.techarts.com> wrote:
Actually, we're letting several of our domain names go back into the field.
I believe there must be some industry way to use some HTML element/tag to
get the Spiders to start updating their links to our main site for the
remainder of the domain's life.
Spiders periodically trawl websites, revisiting ones that they know
about, and should remove dead links, eventually.
I have found this meta element/tag, but that doesn't help with what we want
to accomplish:

<meta name="revisit-after" content="5 days">


You need to reconfigure your server to send appropriate HTTP headers. I
think that you'll find that such meta elements are only paid attention
to by a few clients specifically looking for such things.

For Apache, that's easy enough to do. There are "permanently moved"
headers (a 301 error), that redirects to the new location, and should be
understood to replace any previously indexed address with the new one.
And there's a "gone" header (error 410), which means that the resource
is permanently gone, and has no new location.

You'd have to learn the method to apply that to all the addresses that
you want those sort of responses with (e.g. wildcarding for any resource
requested within a particular domain, or for a directory inside one).

--
My "from" address is totally fake. (Hint: If I wanted e-mails from
complete strangers, I'd have put a real one, there.) Reply to usenet
postings in the same place as you read the message you're replying to.
Jul 20 '05 #3
Were these domains actually being used for content, or were they just
forwarding to other domains?

If they were significant, you may want to consider hanging onto them vs.
losing the traffic.

What I would do:

1. Analyze the actual traffic going to these domains
2. Consider whether it may be worth keeping them (if traffic is
significant)
3. Resubmit the "keeper" sites (I do not believe you can edit spiders or
existing submissions to reflect a new domain name)

- Clint DeBoer
CD Media, Inc.
www.cdmedia.com

"RTL" <bl*******************@nospam.techarts.com> wrote in message
news:LF******************@twister.nyroc.rr.com...
Hello all,

Actually, we're letting several of our domain names go back into the field. I believe there must be some industry way to use some HTML element/tag to
get the Spiders to start updating their links to our main site for the
remainder of the domain's life.

I have found this meta element/tag, but that doesn't help with what we want to accomplish:

<meta name="revisit-after" content="5 days">

I've also gone to alt.spiders, but there were no messages.

Any help would be appreicated!

Rich
www.techarts.com

Jul 20 '05 #4

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

7
by: Colman | last post by:
I couldn't think of a concise way to describe this in the Subject! Anyways... I am creating a PHP/MySQL driven website. I would like to provide other users a sort of customized front-end PHP...
10
by: moondaddy | last post by:
I'm writing an ecommerce app in asp.net/vb.net and need to make the pages searchable and crawlable by spiders, particularly Google's. As far as I know, if my pages's contents are mostly populated...
13
by: Peter Chant | last post by:
I'm considering setting a website up for a club. I do not plan the contents to be for public consumption, but on the other hand I'm not going to have anything on there that is confidential, that...
5
by: Tyler | last post by:
I am developing an application which will allow me to automatically sign into an external website. I can currently do a screen scrape using HTTPWEBREQUEST. However I want to just redirect to the...
2
by: crferguson | last post by:
I'm having a really odd issue. Recently my company has upgraded our data server. For a couple of months I'm having to host two versions of the same website on our webserver until the new data...
21
by: John | last post by:
Hi, I updated a site and changed the file extensions from .html to .php. Now i noticed that the google does find the old .html pages but since they're not there anymore... they can't be found....
3
by: glezaber | last post by:
Hi, I´ve two websites stored in IIS 5.0, and what I want is to Redirect from A website to B website, the problem is that doesn´t work because if I use Response.Redirect (http://localhost:5080)...
5
by: rockdale | last post by:
Hi, all: I have a website with its own login page. Now one of my clients want their employees log into my website from their website. They want to have their login page (look and feel are...
3
by: =?Utf-8?B?c3VyZXNocGFuZGk=?= | last post by:
Hi I am in a web site A. I want to redirect to a web site B with basic authentication. HttpWebContext and WebRespose methods are downloading a site page as html and this is not helping me. I...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: Vimpel783 | last post by:
Hello! Guys, I found this code on the Internet, but I need to modify it a little. It works well, the problem is this: Data is sent from only one cell, in this case B5, but it is necessary that data...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...
1
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
1
by: Defcon1945 | last post by:
I'm trying to learn Python using Pycharm but import shutil doesn't work
1
by: Shællîpôpï 09 | last post by:
If u are using a keypad phone, how do u turn on JavaScript, to access features like WhatsApp, Facebook, Instagram....
0
by: af34tf | last post by:
Hi Guys, I have a domain whose name is BytesLimited.com, and I want to sell it. Does anyone know about platforms that allow me to list my domain in auction for free. Thank you
0
by: Faith0G | last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.