Hi, Does anyone know about robot.txt?
Do I really need one for my site?
If so, how do I get it and what does it do for me?
Thanks for your time,
Mike Pratz http://www.chowardcompany.com
Jul 20 '05
78  6135 
Sometime around Sun, 14 Dec 2003 15:35:23 -0500, Neal is reported to have
stated: "Stan Brown" <th************@fastmail.fm> wrote in message
news:MP************************@news.odyssey.net.. . This isn't rocket surgery.
Just Usenet surgery...
Now, just hold still for a moment. This won't hurt. Much.
--
Mark Parnell http://www.clarkecomputers.com.au
"Neal" <ne**@spamrcn.com> wrote in message
news:3f**********************@news.rcn.com...
"Arthur Pratz" <AP****@worldnet.att.net> wrote in message
news:9d***********************@bgtnsc04-news.ops.worldnet.att.net... Hi, Does anyone know about robot.txt?
Do I really need one for my site?
If so, how do I get it and what does it do for me?
Thanks for your time,
Mike Pratz
http://www.chowardcompany.com
Mike,
A few unsolicited comments.
1) It takes me about 5 minutes to load this page on dialup. Kinda on the
slow side. Background image alone is 120kb. That's just too big.
2) I'm sorry, but marquee is just annoying. It makes the company look like
it's being run by a bunch of 14 year olds.
3) Never heard of the font Maiandra GD, and I'd bet most users don't have
it installed.
4) Look at the labels on the product. This company has a nice
old-fashioned style to it. Why not lose some of the high-tech stuff and keep it simple?
I think it would load quicker, and sell a lot more gum.
5) When the screen has finally loaded, it's an attractive screen, but I
have to scroll before I can do anything. Try to incorporate the links higher up
on the page.
Other than that, I do generally like the page.
Thanks for your comments.
I appreciate your honesty.
There are some pieces of the site that I will be changing,
like the marquee, and the DHTML effects at the top.
I have to work with what the owner of the company wants as well.
Thanks,
Mike
Neal wrote: "William Tasso" wrote
Arthur Pratz wrote:
Before you go off on me, This is my OPINION.
With the greatest of respect, your opinion on this is utterly and
completely irrelevant.
Well, not so fast. His opinion will lead him to not get good
searches on the big engines, and will shut off many visitors from
using the site, but if he values creative design and using the
latest tech over accessibility, that's a valid opinion.
Right. And if a carpenter values colorful shingles and using the
latest nail gun models over proper roofing, that too is a valid
opinion. It only leaves the realm of opinion when the roof starts to
leak.
--
Brian
follow the directions in my address to email me
"Brian" <us*****@julietremblay.com.invalid-remove-this-part> wrote in
message news:T4vDb.564376$Fm2.531842@attbi_s04... Right. And if a carpenter values colorful shingles and using the
latest nail gun models over proper roofing, that too is a valid
opinion. It only leaves the realm of opinion when the roof starts to
leak.
You got it. Hey, if you're looking for that tar gunk, hire someone else,
he's into form, not function.
Stan Brown wrote: But, frankly, I don't believe you. Never a single unwanted pop-up?
What, did you just start Web surfing yesterday?
is the sarcasm really necessary. I'll answer questions that are phrased friendly.
--
Regards,
Ralph
~~
"Alan J. Flavell" <fl*****@ph.gla.ac.uk> wrote Conclusion: I don't need to execute untrusted code retrieved from
random web sites.
"Harlan Messinger" <h.*********@comcast.net> wrote:
Can't you just restrict the execution of code that tries to run ActiveX
controls?
For many people, that's not an easy task. But even just allowing normal
JavaScript (or MSIE's JScript), but ignoring ActiveX, still leaves you
open to its problems (be they malicious or authoring incompetence).
--
My "from" address is totally fake. The reply-to address is real, but
may be only temporary. Reply to usenet postings in the same place as
you read the message you're replying to.
This message was sent without a virus, please delete some files yourself.
Tim wrote: Research into browser exploits, and security flaws for the grotty
details, but even without those things you're subjected to pages which
pop up unrequested windows, pages that open two more windows when you
close one (giving you exponential problems when those other windows do
the same thing). Damn fool scripting that dead-loops, or is just too
CPU intensive, and brings your PC to its knees. Scripting in MSIE
(where they lump Active and JavaScript into the one control) that can
seriously damage your computer's health. There's a large list of
reasons not to allow it unless you have to.
"Ralph Friedman" <ro*********@tneyvafbsgjner.pbz.com> wrote:
I've never disallowed it and never experienced anything of what you describe above.
Count yourself lucky, then. Or do you have some unusual browsing
habits? (Unusual in not what the general public seems to do, or the
software that they use.)
If I were to be suspicious, I'd be inclined to think that you were about
to say that you use a better web browser than MSIE, but decided to play
devil's advocate.
--
My "from" address is totally fake. The reply-to address is real, but
may be only temporary. Reply to usenet postings in the same place as
you read the message you're replying to.
This message was sent without a virus, please delete some files yourself.
"Arthur Pratz" <AP****@worldnet.att.net> wrote in message
news:Yi***********************@bgtnsc04-news.ops.worldnet.att.net... Stan, My name is Mike.
Arthur is my father, obviosly i am using his computer at this time.
It sounds to me that your not a big fan of JavaScript.
Would you agree that there are most likely a larger percent of sites on
the web with Java or other types of code.
I find that it lets the web designer use and show there imagination better
than ordinary text.
I use Javascript for various purposes. That notwithstanding, a web page is
best judged according to how effectively it communicates, not according to
how well it shows off the cleverness of the web designer or allows him to
indulge his creative inclinations.
"Arthur Pratz" <AP****@worldnet.att.net> wrote in message
news:rx***********************@bgtnsc04-news.ops.worldnet.att.net...
"Harlan Messinger" <h.*********@comcast.net> wrote in message
news:br************@ID-114100.news.uni-berlin.de...
"Find Yourself" <nu**@null.com> wrote in message
news:br**********@news-int2.gatech.edu... "Arthur Pratz" <AP****@worldnet.att.net> wrote in message
news:9d***********************@bgtnsc04-news.ops.worldnet.att.net...
> Hi, Does anyone know about robot.txt?
> Do I really need one for my site?
> If so, how do I get it and what does it do for me?
> Thanks for your time,
> Mike Pratz
> http://www.chowardcompany.com
Your UI is wrong on so many levels. At least move the navigation to
the top.
Wow, I'll say. Even at 1280 x 960 I have to maximize my browser to get
the full benefit of the display in one screen. And not even just on the
gateway page.
I view the site at 800X600.
Then you must know what I'm talking about. You have to scroll down once or
twice just to see the graphics, don't you? That's not good.
Tim wrote: Count yourself lucky, then. Or do you have some unusual browsing
habits? (Unusual in not what the general public seems to do, or the
software that they use.)
I have unaskedfor popups suppressed in both Opera 7 and MozillaFirebird 0.7 which
are my everyday browsers.
--
Regards,
Ralph
~~
"Harlan Messinger" <h.*********@comcast.net> wrote in message
news:br************@ID-114100.news.uni-berlin.de...
"Arthur Pratz" <AP****@worldnet.att.net> wrote in message
news:rx***********************@bgtnsc04-news.ops.worldnet.att.net...
"Harlan Messinger" <h.*********@comcast.net> wrote in message
news:br************@ID-114100.news.uni-berlin.de...
"Find Yourself" <nu**@null.com> wrote in message
news:br**********@news-int2.gatech.edu...
> "Arthur Pratz" <AP****@worldnet.att.net> wrote in message
> news:9d***********************@bgtnsc04-news.ops.worldnet.att.net...
> > Hi, Does anyone know about robot.txt?
> > Do I really need one for my site?
> > If so, how do I get it and what does it do for me?
> > Thanks for your time,
> > Mike Pratz
> > http://www.chowardcompany.com
>
> Your UI is wrong on so many levels. At least move the navigation to the > top.
Wow, I'll say. Even at 1280 x 960 I have to maximize my browser to get the full benefit of the display in one screen. And not even just on the
gateway page.
I view the site at 800X600.
Then you must know what I'm talking about. You have to scroll down once or
twice just to see the graphics, don't you? That's not good.
Sorry, I don't understand why that's not good?
I could see scrolling sideways maybe could be annoying.
Mike
"Stan Brown" <th************@fastmail.fm> wrote in message
news:MP************************@news.odyssey.net.. . In article <OMsDb.200650$Ec1.7327785@bgtnsc05-
news.ops.worldnet.att.net> in comp.infosystems.www.authoring.html,
Arthur Pratz wrote:There are some pieces of the site that I will be changing,
like the marquee, and the DHTML effects at the top.
But not the broken navigation, I infer?
I have to work with what the owner of the company wants as well.
Are you telling us that the owner wants to turn away millions of
potential customers, _and_ make pages impossible to find through
Google?
Are you sure the owner is aware of the implications of the broken
design you are doing?
Stan, the navigation is only broken because you cant view JS.
However, I am thinking about what you and others in the group have been
saying.
I was thinking about making a text link on the homepage that says to click
here if you are having trouble viewing this site.
That link could be a duplicate of the entire site in text and graphics only
format.
What is your thought on that?
Mike
On Tue, 16 Dec 2003 23:37:34 GMT, Arthur Pratz declared in
comp.infosystems. www.authoring.html: Stan, the navigation is only broken because you cant view JS.
Along with ~10-15% of your visitors, as well as Googlebot.
However, I am thinking about what you and others in the group have been
saying.
I was thinking about making a text link on the homepage that says to click
here if you are having trouble viewing this site.
"Click here" isn't appropriate link text, but that's another issue.
That link could be a duplicate of the entire site in text and graphics only
format.
What is your thought on that?
Well, you could double your work if you really want to. Or you could use
navigation that works in the first place.
--
Mark Parnell http://www.clarkecomputers.com.au
"Arthur Pratz" <AP****@worldnet.att.net> wrote in
news:2H***********************@bgtnsc04-news.ops.worldnet.att.net: Stan, the navigation is only broken because you cant view JS.
However, I am thinking about what you and others in the group have
been saying.
I was thinking about making a text link on the homepage that says to
click here if you are having trouble viewing this site.
That link could be a duplicate of the entire site in text and graphics
only format.
What is your thought on that?
Double the work for very little if any gain (every time you change your
site, you have to make the change in two places, and the two versions will
almost certainly get out of sync).
Design your navigation so that if JS isn't enabled (as it won't be for a
search engine or link-checking program), it will still work even if it
isn't as flashy. For example, you could mark up a navigation menu as a
simple nested list of links. If JS isn't available, it will look pretty
plain, even ugly, but your JS code, if it runs, can take that list and
restyle it into a flashy-looking drop-down menu system.
If you do it right, you'll have a piece of reusable JS code that you can
simply drop into *any* of your pages without having to rewrite it for
different menus. To change the links, you just change the markup, not the
code (which, among other things, means that you can put the code in a
cacheable external file, rather than duplicating it on each page, thus
reducing your server's bandwidth load). So a lot of up-front learning
followed by a moderate amount of up-front work can save you a lot of time
and effort down the road.
The idea is that you use plain old HTML markup to create your site's
infrastructure, and then use JS and CSS to create your site's facade. And
if you separate them correctly, then it becomes extraordinarily easy to
make changes to your site's look and feel without having to do a lot of
rework. And that, in turn, makes it easy to experiment with your site's
look and feel, which in turn makes it easy to *optimize* your site's look
and feel for the majority of users without compromising its usability to
anyone.
"Eric Bohlman" <eb******@earthlink.net> wrote in message
news:Xn*******************************@130.133.1.4 ... "Arthur Pratz" <AP****@worldnet.att.net> wrote in
news:2H***********************@bgtnsc04-news.ops.worldnet.att.net:
Stan, the navigation is only broken because you cant view JS.
However, I am thinking about what you and others in the group have
been saying.
I was thinking about making a text link on the homepage that says to
click here if you are having trouble viewing this site.
That link could be a duplicate of the entire site in text and graphics
only format.
What is your thought on that?
Double the work for very little if any gain (every time you change your
site, you have to make the change in two places, and the two versions will
almost certainly get out of sync).
Design your navigation so that if JS isn't enabled (as it won't be for a
search engine or link-checking program), it will still work even if it
isn't as flashy. For example, you could mark up a navigation menu as a
simple nested list of links. If JS isn't available, it will look pretty
plain, even ugly, but your JS code, if it runs, can take that list and
restyle it into a flashy-looking drop-down menu system.
If you do it right, you'll have a piece of reusable JS code that you can
simply drop into *any* of your pages without having to rewrite it for
different menus. To change the links, you just change the markup, not the
code (which, among other things, means that you can put the code in a
cacheable external file, rather than duplicating it on each page, thus
reducing your server's bandwidth load). So a lot of up-front learning
followed by a moderate amount of up-front work can save you a lot of time
and effort down the road.
The idea is that you use plain old HTML markup to create your site's
infrastructure, and then use JS and CSS to create your site's facade. And
if you separate them correctly, then it becomes extraordinarily easy to
make changes to your site's look and feel without having to do a lot of
rework. And that, in turn, makes it easy to experiment with your site's
look and feel, which in turn makes it easy to *optimize* your site's look
and feel for the majority of users without compromising its usability to
anyone.
That actually sounds good, but I don't know how to write JS code.
I have been using free scripts from the web.
Mike
"Arthur Pratz" <AP****@worldnet.att.net> wrote in
news:ju***********************@bgtnsc04-news.ops.worldnet.att.net: That actually sounds good, but I don't know how to write JS code.
An eminently curable condition. Seek out some JS tutorials on the Web (but
watch out for poor-quality tutorials), read comp.lang.javascript and its
FAQ, and/or buy a copy of David Flanagan's _javascript: The Definitive
Guide_ from O'Reilly.
I have been using free scripts from the web.
Throwing stuff that you don't fully understand into Web pages is not the
route to quality. Admittedly, lots of Web designers start out by doing
that, but the top-notch ones quickly move beyond it.
Once again, you'll find that the up-front effort to fully understand what
you're working with will be paid back many times over.
Tim wrote: Count yourself lucky, then. Or do you have some unusual browsing
habits? (Unusual in not what the general public seems to do, or the
software that they use.)
"Ralph Friedman" <ro*********@tneyvafbsgjner.pbz.com> wrote:
I have unaskedfor popups suppressed in both Opera 7 and MozillaFirebird 0.7 which
are my everyday browsers.
So, you do have what most (*) people would refer to as unusual browsing
habits (using browsers that most people have never even heard of, nor do
most people reconfigure their browsers).
* I'll qualify that definition of most by saying that it's everyone that
I have observed. Even when suggesting they reconfigure such options,
they object saying that they can't be bothered, don't want to cripple
some sites, and have no idea what it's about.
--
My "from" address is totally fake. The reply-to address is real, but
may be only temporary. Reply to usenet postings in the same place as
you read the message you're replying to.
This message was sent without a virus, please delete some files yourself.
"Arthur Pratz" <AP****@worldnet.att.net> wrote in message
news:VA***********************@bgtnsc04-news.ops.worldnet.att.net...
"Harlan Messinger" <h.*********@comcast.net> wrote in message
news:br************@ID-114100.news.uni-berlin.de...
"Arthur Pratz" <AP****@worldnet.att.net> wrote in message
news:rx***********************@bgtnsc04-news.ops.worldnet.att.net...
"Harlan Messinger" <h.*********@comcast.net> wrote in message
news:br************@ID-114100.news.uni-berlin.de...
>
> Wow, I'll say. Even at 1280 x 960 I have to maximize my browser to
get the > full benefit of the display in one screen. And not even just on the
gateway
> page.
>
I view the site at 800X600.
Then you must know what I'm talking about. You have to scroll down once
or twice just to see the graphics, don't you? That's not good.
Sorry, I don't understand why that's not good?
I could see scrolling sideways maybe could be annoying.
Mike
For the same reason it would be tough to appreciate the Mona Lisa you could
only see a third of it at a time.
On 15 Dec 2003 15:52:40 GMT, "Ralph Friedman"
<ro*********@tneyvafbsgjner.pbz.com> wrote: Tim wrote: Damn fool scripting that dead-loops, or is just too
CPU intensive, and brings your PC to its knees.
Most browsers have always protected against this - IE pops up a
message saynig "a script on this page has been running for too long" -
of course it's still possible to max out cpu without doing that, but
dead loops shouldn't exist.
Scripting in MSIE
(where they lump Active and JavaScript into the one control) that can
seriously damage your computer's health.
I'm not sure what you mean by "Active" and "JavaScript" into one
control - What do you understand ActiveScripting? It's just MS's name
for their script technology solution in IE which allows you to script
pages with perlscript, jscript, vbscript etc. - they can't use a
control saying "disable javascript" - it's a trademark of Sun.
I've never disallowed it and never experienced anything of what you =
describe above.
You've been lucky once, it doesn't mean you'll be again.
It's generally reasonably safe to run a patched browser with script
enabled if you stay in the high google page-rank/respected site world,
but straying beyond that could well start hurting.
Jim.
--
comp.lang.javascript FAQ - http://jibbering.com/faq/
On Mon, 15 Dec 2003 18:44:26 +0000, "Alan J. Flavell"
<fl*****@ph.gla.ac.uk> wrote: In one, due to mistyping of a URL, I happened upon a compromised web
site which promptly attempted to use Javascripting to install a
"premium rate dialler" on Windows for me.
I feel this is slightly unfair, javascript was wholly unnecessary for
that attempt, and disabling it should've done nothing to protect you
(of course the author was probably a typical incompetent relying on
script as she knew no better)
Jim.
--
comp.lang.javascript FAQ - http://jibbering.com/faq/
Jim Ley wrote: Scripting in MSIE (where they lump Active and JavaScript into
the one control) that can seriously damage your computer's
health.
I'm not sure what you mean by "Active" and "JavaScript" into one
control - What do you understand ActiveScripting? It's just MS's
name for their script technology solution in IE which allows you to
script pages with perlscript, jscript, vbscript etc. - they can't
use a control saying "disable javascript" - it's a trademark of
Sun.
I have 2 separate settings on my win2k machine: active scripting and
ActiveX controls. I assume they mean different things.
--
Brian
follow the directions in my address to email me
Jim Ley wrote: You've been lucky once, it doesn't mean you'll be again.
it's been a bit more than once.
It's generally reasonably safe to run a patched browser with script
enabled if you stay in the high google page-rank/respected site world,
but straying beyond that could well start hurting.
don't tend to travel in those circles.
--
Regards,
Ralph
~~
On Thu, 18 Dec 2003, Jim Ley wrote: On Mon, 15 Dec 2003 18:44:26 +0000, "Alan J. Flavell"
<fl*****@ph.gla.ac.uk> wrote:
In one, due to mistyping of a URL, I happened upon a compromised web
site which promptly attempted to use Javascripting to install a
"premium rate dialler" on Windows for me.
I feel this is slightly unfair,
I hear what you say. I thought I had been asked for personal
experiences, and so I confined myself to that: anyone can go and read
javascript compromises reported at various security information sites
if they think my sample is untypical and want a wider view.
javascript was wholly unnecessary for that attempt, and disabling it
should've done nothing to protect you (of course the author was
probably a typical incompetent [..]
Could I draw your attention to this:
| It's generally reasonably safe to run a patched browser with script
| enabled if you stay in the high google page-rank/respected site
| world, but straying beyond that could well start hurting.
As I mentioned above, there are malicious web sites out there which
are deliberately set up to catch typos. Even bona fide web servers
have been compromised (don't I know it? - several years back I was the
server admin of a server that got broken into and mischievous material
planted on it).
It only takes one rotten apple to spoil the whole barrel, as the
saying goes.
Alan J. Flavell wrote: It only takes one rotten apple to spoil the whole barrel, as the
saying goes.
if that's the case, the only "secure" solution would be browsing with telnet.
--
Regards,
Ralph
~~
On Fri, 19 Dec 2003, Ralph Friedman wrote: Alan J. Flavell wrote:
It only takes one rotten apple to spoil the whole barrel, as the
saying goes.
if that's the case, the only "secure" solution would be browsing with telnet.
You better make sure you've got a 100% dumb terminal, if you think
that's totally secure. SCNR.
But, putting any attempts at sarcasm aside: surely the point here is
that this is an HTML _authoring_ group? If there's an available
solution to our authoring intentions which more of our readers will
accept as secure, then I'd recommend choosing it. If I wanted to
discuss the underlying issue, then I'd go looking for a relevant
security group...
When I discuss enabling or not enabling javascript in this forum, I
don't mean it in any sense as a generic javascript advocacy argument,
but no more than to exemplify that there _are_ issues, which our more
discerning readers may very well have taken into account in setting
their browsing preferences.
In that sense, we rate to reach more of them if we don't attempt to
mandate something that they've turned off. Indeed, they might be more
inclined to turn it on again for us if we offer it to them as an
optional convenience, than if we try to browbeat them into doing
something that they already decided was questionable.
cheers This thread has been closed and replies have been disabled. Please start a new discussion. Similar topics
by: Adrian Lumsden |
last post by:
Hello,
I have an app where I have to screen scrape to capture an image from a JMF
film player.
The user is given a dialog with a list of frames that can be exported as
images. If the one they...
|
by: Christopher Koppler |
last post by:
I need to simulate mouseclicks (in Win2K) in another application's
window. Googling threw up java.awt.Robot, and a bit of trial and error
showed that it could do what I needed (simulate a...
|
by: Robert |
last post by:
I have python related stuff on some of my web pages.
This month "The Python Robot" is going for over 400 MB / 4000 accesses
downloads in my stats ! increasing frequency the last months!? thats...
|
by: Jonathan Vance |
last post by:
I am looking for a python robot that Van Rossum released with python
0.9.8. It may have been the first web robot (see
http://www.webhistory.org/www.lists/www-talk.1993q1/0060.html).
I've had no...
|
by: Marcelo |
last post by:
Hi,
I need to make a robot that fills forms. I already made that with php
sites, but now I need that in a javascript pages site ( the robot can
be php,asp, asp.net ), anyone knows how can I do...
|
by: nnobakht |
last post by:
Hi, I'm working on an assignment for school which i am a bit stuck on. The assignment is to make robot which i have been given the library for move around different boards and collecting "coins" and...
|
by: Shiv Kumar |
last post by:
Rational Robot is a complete set of components for automating the
testing of Microsoft Windows client/server and Internet applications
running under Windows NT 4.0, Windows XP, Windows 2000, and...
|
by: socialanxiety |
last post by:
i hope someone here can help me.
basically, me and my friend have a summer project.
in this project, we need something that would basically function as a
blender. we know we'll need to buy a...
|
by: John Nagle |
last post by:
I just discovered that the "robotparser" module interprets
a 403 ("Forbidden") status on a "robots.txt" file as meaning
"all access disallowed". That's unexpected behavior.
A major site...
|
by: isladogs |
last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM).
In this session, we are pleased to welcome former...
|
by: ryjfgjl |
last post by:
In our work, we often need to import Excel data into databases (such as MySQL, SQL Server, Oracle) for data analysis and processing. Usually, we use database tools like Navicat or the Excel import...
|
by: Charles Arthur |
last post by:
How do i turn on java script on a villaon, callus and itel keypad mobile phone
|
by: aa123db |
last post by:
Variable and constants
Use var or let for variables and const fror constants.
Var foo ='bar';
Let foo ='bar';const baz ='bar';
Functions
function $name$ ($parameters$) {
}
...
|
by: ryjfgjl |
last post by:
If we have dozens or hundreds of excel to import into the database, if we use the excel import function provided by database editors such as navicat, it will be extremely tedious and time-consuming...
|
by: emmanuelkatto |
last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud.
Please let me know.
Thanks!
Emmanuel
|
by: BarryA |
last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
|
by: nemocccc |
last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
|
by: Sonnysonu |
last post by:
This is the data of csv file
1 2 3
1 2 3
1 2 3
1 2 3
2 3
2 3
3
the lengths should be different i have to store the data by column-wise with in the specific length.
suppose the i have to...
| |
