473,404 Members | 2,137 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,404 software developers and data experts.

Using sed to strip hex x80 through xFF

114 100+
have a ASCII file that is messed up. THe problem is that unprintable characters got into this text file. So when printing or using scripts to work with this file I get errors.

I need to clean up this file removing all hex 80 - FF from this file. Been told that I need to use sed in a script to do this. Have looked for examples but can not find anything dealing with what I need to do.

Could someone help me with the sed command that I would need to clean this file. I would like to create a new file from the old so that I have the old as a backup just in case.

Thanks
Jan 12 '07 #1
5 13643
ghostdog74
511 Expert 256MB
have a ASCII file that is messed up. THe problem is that unprintable characters got into this text file. So when printing or using scripts to work with this file I get errors.

I need to clean up this file removing all hex 80 - FF from this file. Been told that I need to use sed in a script to do this. Have looked for examples but can not find anything dealing with what I need to do.

Could someone help me with the sed command that I would need to clean this file. I would like to create a new file from the old so that I have the old as a backup just in case.

Thanks
would you want to give a sample of that file?
Jan 14 '07 #2
ljungers
114 100+
The file is a regular text file wherre hex 00 - 1F are control characters, hex 20 - 7E are printable characters and hex 7F is DEL chharacter.

Some how hex (unprintable) characters got into this file (I think that this file was a print file at one time0 like hex characters B0, CA, 80, 83 are one I have found so far.

I need to read this file and output a files with all hex 80 and above removed from this file so that other scripts can run and use this file without causing errors in the script.

Thanks for any help you can give me.
Jan 14 '07 #3
ghostdog74
511 Expert 256MB
i can try to help, but please provide a sample. Its easy to strip x80, xFF but i just want to make sure i don't strip useful data.
Jan 15 '07 #4
Garmo
1
i can try to help, but please provide a sample. Its easy to strip x80, xFF but i just want to make sure i don't strip useful data.
I need a variation of this command. I'd like to use sed to read a file with no line feeds, and replace a particular sequence with the same sequence preceded by a line feed. This is UNIX. This is what I've tried so far.

sed 's/ISA/\x'012'ISA/g' filename

I'm stuck on the syntax for adding a UNIX line feed. Any ideas??

A typical file would look something like this:

ISAklsdfsdfgksdfgkjsdfgjdfgsdfgkdISAl;kasdfkasdfka sdfk
Apr 10 '07 #5
ghostdog74
511 Expert 256MB
I need a variation of this command. I'd like to use sed to read a file with no line feeds, and replace a particular sequence with the same sequence preceded by a line feed. This is UNIX. This is what I've tried so far.

sed 's/ISA/\x'012'ISA/g' filename

I'm stuck on the syntax for adding a UNIX line feed. Any ideas??

A typical file would look something like this:

ISAklsdfsdfgksdfgkjsdfgjdfgsdfgkdISAl;kasdfkasdfka sdfk
for line feed, type ctrl-v then 'm'. NOT shift-6 then 'm'. something like this
Expand|Select|Wrap|Line Numbers
  1. sed 's/ISA/^MISA/g' file > file1
  2.  
Apr 11 '07 #6

Sign in to post your reply or Sign up for a free account.

Similar topics

4
by: bmiras | last post by:
I've got a problem using urllib2 to get a web page. I'm going through a proxy using user/password authentification and i'm trying to get a page asking for a HTTP authentification. And I'm using...
0
by: fishboy | last post by:
Howdy, Sorry if this is a double post. First try seemed to go into hyperspace. I'm working on a personal project. It's going to be a multipart binary attachment downloader that will search...
10
by: Julian Smith | last post by:
I've been playing with a function that creates an anonymous function by compiling a string parameter, and it seems to work pretty well: def fn( text): exec 'def foo' + text.strip() return foo ...
6
by: Mark Miller | last post by:
I have a scheduled job that uses different XSL templates to transform XML and save it to disk. I am having problems with the code below. The problem shows up on both my development machine (Windows...
6
by: rtilley | last post by:
s = ' qazwsx ' # How are these different? print s.strip() print str.strip(s) Do string objects all have the attribute strip()? If so, why is str.strip() needed? Really, I'm just curious......
6
by: eight02645999 | last post by:
hi can someone explain strip() for these : 'example' when i did this: 'abcd,words.words'
2
by: =?iso-8859-1?b?cultaQ==?= | last post by:
Hi, I would like to rename files (jpg's ones) using a text file containing the new names... Below is the code that doesn't work : ***** #!/usr/bin/python #-*- coding: utf-8 -*- from os...
3
by: Colin J. Williams | last post by:
The Library Reference has strip( ) Return a copy of the string with the leading and trailing characters removed. The chars argument is a string specifying the set of characters to be removed....
0
by: Kam-Hung Soh | last post by:
On Sun, 18 May 2008 00:47:55 +1000, Beema shafreen <beema.shafreen@gmail.comwrote: 1. Test with a small number of short files with a clear idea of the expected result. 2. Use better...
6
by: Christoph Zwerschke | last post by:
In Python programs, you will quite frequently find code like the following for removing a certain prefix from a string: if url.startswith('http://'): url = url Similarly for stripping...
0
by: emmanuelkatto | last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud. Please let me know. Thanks! Emmanuel
0
BarryA
by: BarryA | last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
1
by: nemocccc | last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
0
by: Hystou | last post by:
There are some requirements for setting up RAID: 1. The motherboard and BIOS support RAID configuration. 2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...
0
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However,...
0
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can...
0
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows...
0
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing,...
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.