473,699 Members | 2,496 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

Question about ReadLine UTF8 line truncation

(Dot Net 2 C# application - using Encoding.UTF8 with a StreamReader)
I have a very strange problem that I cannot explain with a UTF8 Readline()
although this could exist in other types of encoding, I have not tried them.

Our application wrote this sequence to a UTF8 file. Now I am loading it
back and the text is not coming back in the same as it went out.

DATA:
from: processfrom checkemail failed: 501 syntax error in parameters: invalid
char in email: "sometext\conte nt-transfer-encoding:"@serv er.com
command: mail from:"sometext\ content-transfer-encoding:"@serv er.com
Each of those lines will be slipt at the \c of the
content-transfer-encoding. They are not high order characters, and I don't
ever remember \c being a control character for anything.
So instead of getting back in two lines that I wrote out, I get in 4 lines.

Any ideas as to why this is, and how would I correct it? I am guessing that
I will need to escape the sequence prior to writing it out, but I don't know
why and that bugs me.

Thanks for any help.

Feb 12 '07 #1
4 4886
EmeraldShield <em***********@ noemail.noemail wrote:
(Dot Net 2 C# application - using Encoding.UTF8 with a StreamReader)
I have a very strange problem that I cannot explain with a UTF8 Readline()
although this could exist in other types of encoding, I have not tried them.
<snip>

Could you post a short but complete program which demonstrates the
problem?

See http://www.pobox.com/~skeet/csharp/complete.html for details of
what I mean by that.

--
Jon Skeet - <sk***@pobox.co m>
http://www.pobox.com/~skeet Blog: http://www.msmvps.com/jon.skeet
If replying to the group, please do not mail me too
Feb 12 '07 #2
Hello Jon,

Based on your description, when you use StreamReader with UTF8 encoding to
read some text data writen out previously, you get the wrong
string(differen t from original output), correct?

Would you show to simple test code snippet on this so that we can get a
more clear view on your code logic? Also, as for the following test
fragment you mentioned:

==============
from: processfrom checkemail failed: 501 syntax error in parameters:
invalid char in email:
"sometext\conte nt-transfer-encoding:"@serv er.comcommand: mail
from:"sometext\ content-transfer-encoding:"@serv er.com
==============

Did you directly embeded in C# code like

string txt = " ..... the text here.....";

or is it load from some other source(such as a Textbox or from another
file)? Also, when you output the data to the txt file(through
StreamReader+UT F8 encoding), have you checked the txt output file to see
whether the output is correctly expected?

Based on my experience, such problem is likely occur when you directly
embeded the string in code since there are some particular chars that need
escaping in C# string. For example, you need to escape \ as \\ So if
you directly embed string in C# code, you need to escape the whole string
as below:

=============
string txt = "from: processfrom checkemail failed: 501 syntax error in
parameters: invalid char in email:
\"sometext\\con tent-transfer-encoding:\"@ser ver.com command: mail
from:\"sometext \\content-transfer-encoding:\"@ser ver.com";

StreamWriter sw = new StreamWriter("d irect_output.tx t", false,
Encoding.UTF8);
sw.Write(txt);

sw.Close();

===============

In addition, I suggest you put those string in a TextBox and writeout it
from that TextBox into the StreanWriter to see whether the output is as
expected. here is my test code which works well for the text fragment you
provided.

=============== ==============
private void btnSave_Click(o bject sender, EventArgs e)
{
StreamWriter sw = new StreamWriter("o utput.txt", false,
Encoding.UTF8);
sw.Write(textBo x1.Text);

sw.Close();
}

private void btnLoad_Click(o bject sender, EventArgs e)
{
StreamReader sr = new StreamReader("o utput.txt", Encoding.UTF8);

string txt = sr.ReadToEnd();

sr.Close();
MessageBox.Show (txt);
}
=============== ========

Please feel free to post here if there is anything unclear or if you met
any furtehr problems.

Sincerely,

Steven Cheng

Microsoft MSDN Online Support Lead

=============== =============== =============== =====

Get notification to my posts through email? Please refer to
http://msdn.microsoft.com/subscripti...ult.aspx#notif
ications.

Note: The MSDN Managed Newsgroup support offering is for non-urgent issues
where an initial response from the community or a Microsoft Support
Engineer within 1 business day is acceptable. Please note that each follow
up response may take approximately 2 business days as the support
professional working with you may need further investigation to reach the
most efficient resolution. The offering is not appropriate for situations
that require urgent, real-time or phone-based interactions or complex
project analysis and dump analysis issues. Issues of this nature are best
handled working with a dedicated Microsoft Support Engineer by contacting
Microsoft Customer Support Services (CSS) at
http://msdn.microsoft.com/subscripti...t/default.aspx.

=============== =============== =============== =====

This posting is provided "AS IS" with no warranties, and confers no rights.

Feb 12 '07 #3
Based on your description, when you use StreamReader with UTF8 encoding to
read some text data writen out previously, you get the wrong
string(differen t from original output), correct?

Would you show to simple test code snippet on this so that we can get a
more clear view on your code logic? Also, as for the following test
fragment you mentioned:
I will do that tonight or tomorrow.
Did you directly embeded in C# code like

string txt = " ..... the text here.....";

or is it load from some other source(such as a Textbox or from another
file)? Also, when you output the data to the txt file(through
StreamReader+UT F8 encoding), have you checked the txt output file to see
whether the output is correctly expected?
No, this is data sent to my application from a remote system over a socket.
I have looked at the output file and it is correct. I have loaded it in
Wordpad, Notepad, and VS2005 and it looks correct in all of them.
In addition, I suggest you put those string in a TextBox and writeout it
from that TextBox into the StreanWriter to see whether the output is as
expected. here is my test code which works well for the text fragment you
provided.
I will try that and see what it looks like. I just wanted to make sure that
the \C was not some sort of escape character that I didn't know about. It
is being sent to my app that way and written straight to disk without any
escaping.

Jason
Feb 12 '07 #4
Thanks for your reply Jason,

Sure, we'll wait for your further update.

Also, I'm sure that "\C" won't be particular escaped by UTF8 encoding or
other encoding readers. Generally, only when we put string directly in code
or some script should we take care of some escaping issue.

Anyway, please feel free to post here if you meet any further problem on
this.

Sincerely,

Steven Cheng

Microsoft MSDN Online Support Lead
This posting is provided "AS IS" with no warranties, and confers no rights.

Feb 13 '07 #5

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

3
8702
by: peter leonard | last post by:
Hi, I having a problem with reading each line from a text file. For example, the file is a text file named 'test.txt' with the following content : line 1 line 2 line 3 line 4 line 5
2
3355
by: Pernell Williams | last post by:
Hi all: Thank you for your responses. I have a more specific question about "file.seek() and file.readline()" versus "file.seek() and file.xreadlines". When I have the following code:
4
1560
by: Paul Rubin | last post by:
I have a file with contents like: Vegetable: spinach Fruit: banana Flower: Daisy Fruit: pear
5
6908
by: Richard Lewis | last post by:
Hi there, I'm having a problem with unicode files and ftplib (using Python 2.3.5). I've got this code: xml_source = codecs.open("foo.xml", 'w+b', "utf8") #xml_source = file("foo.xml", 'w+b') ftp.retrbinary("RETR foo.xml", xml_source.write)
2
1913
by: Macisu | last post by:
Hi I am using the object Stream Reader to read Files. The method ReadLine() does not read the character ñ o Ñ. What Can i do? Helpme please thanks
3
4141
by: Ian Taite | last post by:
Hello, I'm exploring why one of my C# .NET apps has "high" memory usage, and whether I can reduce the memory usage. I have an app that wakes up and processes text files into a database periodically. What happens, is that the app reads the contents of a text file line by line into an ArrayList. Each element of the ArrayList is a string representing a record from the file. The ArrayList is then processed, and the arraylist goes out of...
0
4713
by: 7stud | last post by:
Hi, 1) Does this make any sense: """ Thus, the loop: for line in f: iterates on each line of the file. Due to buffering issues,
6
1178
by: asif08 | last post by:
The following program is giving an error: System.NullReferenceException Object Reference not set to an instance of an object. at IgnoreKwic.GetAllLinesForIndexWords<with parameters> at IgnoreKwic.BuildOutput<with parameters> at IgnoreKwic.Main<> In the line where I wrote that I want to display the output on the screen, would this work? using System.Collections.Generic;// Adds definition of List<T>
6
3734
by: Sean Davis | last post by:
I have a large file that I would like to transform and then feed to a function (psycopg2 copy_from) that expects a file-like object (needs read and readline methods). I have a class like so: class GeneInfo(): def __init__(self): #urllib.urlretrieve('ftp://ftp.ncbi.nih.gov/gene/DATA/ gene_info.gz',"/tmp/gene_info.gz")
1
8941
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
0
8896
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
0
7784
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own.... Now, this would greatly impact the work of software developers. The idea...
1
6546
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms. Adolph will...
0
4390
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
0
4637
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
1
3071
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
2
2362
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.
3
2015
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.