473,473 Members | 1,511 Online
Bytes | Software Development & Data Engineering Community
Create Post

Home Posts Topics Members FAQ

XMLHTTP: translating entities like curly quotes and curly apostrophes?

I'm using XMLHTTP to screen-scrape many thousands of pages of content as
part of a data-structuring project.

One issue that I'm running into is that some entities such as curly quotes
and curly apostrophes do not translate properly; they're returned as
question marks indicating an unidentified character.

I'm guessing the usual hack of writing a translate function doesn't work
since the problem lies in the data being pulled down by XMLHTTP.

Is there anything that can be done, short of using a different
screen-scraping component? I intially used something called "ASPTear", but
moved to XMLHTTP since it seems to return fewer errors in production

Thanks,
-KF
Jul 19 '05 #1
2 1786
"curly quotes"

Let their be light.


"Ken Fine" <ke*****@u.washington.edu> wrote in message
news:c1**********@nntp6.u.washington.edu...
I'm using XMLHTTP to screen-scrape many thousands of pages of content as
part of a data-structuring project.

One issue that I'm running into is that some entities such as curly quotes
and curly apostrophes do not translate properly; they're returned as
question marks indicating an unidentified character.

I'm guessing the usual hack of writing a translate function doesn't work
since the problem lies in the data being pulled down by XMLHTTP.

Is there anything that can be done, short of using a different
screen-scraping component? I intially used something called "ASPTear", but
moved to XMLHTTP since it seems to return fewer errors in production

Thanks,
-KF


Jul 19 '05 #2
In the realm of the useful: best solution I found was to acquire the content
to a local source and search/replace all of the entities to more standard
equivalents. Maybe this helps someone.

"name" <no****@user.com> wrote in message
news:uN***************@TK2MSFTNGP11.phx.gbl...
"curly quotes"

Let their be light.


"Ken Fine" <ke*****@u.washington.edu> wrote in message
news:c1**********@nntp6.u.washington.edu...
I'm using XMLHTTP to screen-scrape many thousands of pages of content as
part of a data-structuring project.

One issue that I'm running into is that some entities such as curly quotes and curly apostrophes do not translate properly; they're returned as
question marks indicating an unidentified character.

I'm guessing the usual hack of writing a translate function doesn't work
since the problem lies in the data being pulled down by XMLHTTP.

Is there anything that can be done, short of using a different
screen-scraping component? I intially used something called "ASPTear", but moved to XMLHTTP since it seems to return fewer errors in production

Thanks,
-KF

Jul 19 '05 #3

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

6
by: Kim Gjerstad | last post by:
I recently changed my web.config file to iso 8859-1 for the request and response encoding. This created another problem: my forms storing information with data type varchar suddenly replace all...
18
by: Colin Steadman | last post by:
My ASP page allows user to enter comments into a form. To avoid errors I'm having to strip out double quotes before saving to the database. Is there anyway to encode these so that I can store...
23
by: gregf | last post by:
I have a paragraph of text pasted into a word document, it's in Polish, complete with polish characters. They show up just fine in word, but the program I use for web page programming, HomeSite,...
3
by: Richard Hollenbeck | last post by:
I have an email field and a command button to send email. The problem is that when the email program opens up there are apostrophes around the email address that I have to manually remove before...
7
by: ian ward | last post by:
Hello, I want to show the result of some JavaScript on an HTML page. I've had a look at some threads and it seems the conclusion is the following - javascript entities are usable on the...
4
by: Ryan | last post by:
Sometimes I have a user who will copy and paste something from Word and it includes a curly variation of an apostrophe. When this text is displayed back to the user at a later date (after pulling...
23
by: Haines Brown | last post by:
I tried changing font-family from helvetica to arial and discovered that I loose typographic quotes, etc. (for example, if I have “ in a page, the double opening quotation mark is not curly). ...
5
by: A.Martini | last post by:
Hello, I have to parse an xml file which has entities like &8220; ” and apostrophes ('). I have created a parser and I parse the xml but the xml_parse function convert entities and apostrophes to...
3
by: bsagert | last post by:
Some web feeds use decimal character entities that seem to confuse Python (or me). For example, the string "doesn't" may be coded as "doesn’t" which should produce a right leaning apostrophe....
0
by: Hystou | last post by:
There are some requirements for setting up RAID: 1. The motherboard and BIOS support RAID configuration. 2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...
0
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However,...
0
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven...
0
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each...
0
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing,...
1
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new...
0
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and...
0
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The...
0
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated ...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.