473,836 Members | 2,240 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

Using sed to strip hex x80 through xFF

114 New Member
have a ASCII file that is messed up. THe problem is that unprintable characters got into this text file. So when printing or using scripts to work with this file I get errors.

I need to clean up this file removing all hex 80 - FF from this file. Been told that I need to use sed in a script to do this. Have looked for examples but can not find anything dealing with what I need to do.

Could someone help me with the sed command that I would need to clean this file. I would like to create a new file from the old so that I have the old as a backup just in case.

Thanks
Jan 12 '07 #1
5 13691
ghostdog74
511 Recognized Expert Contributor
have a ASCII file that is messed up. THe problem is that unprintable characters got into this text file. So when printing or using scripts to work with this file I get errors.

I need to clean up this file removing all hex 80 - FF from this file. Been told that I need to use sed in a script to do this. Have looked for examples but can not find anything dealing with what I need to do.

Could someone help me with the sed command that I would need to clean this file. I would like to create a new file from the old so that I have the old as a backup just in case.

Thanks
would you want to give a sample of that file?
Jan 14 '07 #2
ljungers
114 New Member
The file is a regular text file wherre hex 00 - 1F are control characters, hex 20 - 7E are printable characters and hex 7F is DEL chharacter.

Some how hex (unprintable) characters got into this file (I think that this file was a print file at one time0 like hex characters B0, CA, 80, 83 are one I have found so far.

I need to read this file and output a files with all hex 80 and above removed from this file so that other scripts can run and use this file without causing errors in the script.

Thanks for any help you can give me.
Jan 14 '07 #3
ghostdog74
511 Recognized Expert Contributor
i can try to help, but please provide a sample. Its easy to strip x80, xFF but i just want to make sure i don't strip useful data.
Jan 15 '07 #4
Garmo
1 New Member
i can try to help, but please provide a sample. Its easy to strip x80, xFF but i just want to make sure i don't strip useful data.
I need a variation of this command. I'd like to use sed to read a file with no line feeds, and replace a particular sequence with the same sequence preceded by a line feed. This is UNIX. This is what I've tried so far.

sed 's/ISA/\x'012'ISA/g' filename

I'm stuck on the syntax for adding a UNIX line feed. Any ideas??

A typical file would look something like this:

ISAklsdfsdfgksd fgkjsdfgjdfgsdf gkdISAl;kasdfka sdfkasdfk
Apr 10 '07 #5
ghostdog74
511 Recognized Expert Contributor
I need a variation of this command. I'd like to use sed to read a file with no line feeds, and replace a particular sequence with the same sequence preceded by a line feed. This is UNIX. This is what I've tried so far.

sed 's/ISA/\x'012'ISA/g' filename

I'm stuck on the syntax for adding a UNIX line feed. Any ideas??

A typical file would look something like this:

ISAklsdfsdfgksd fgkjsdfgjdfgsdf gkdISAl;kasdfka sdfkasdfk
for line feed, type ctrl-v then 'm'. NOT shift-6 then 'm'. something like this
Expand|Select|Wrap|Line Numbers
  1. sed 's/ISA/^MISA/g' file > file1
  2.  
Apr 11 '07 #6

Sign in to post your reply or Sign up for a free account.

Similar topics

4
4465
by: bmiras | last post by:
I've got a problem using urllib2 to get a web page. I'm going through a proxy using user/password authentification and i'm trying to get a page asking for a HTTP authentification. And I'm using python 2.3 Here is an exemple of the piece of code I use: import urllib2 #Proxy handler proxy_handler = urllib2.ProxyHandler({"http" :
0
1874
by: fishboy | last post by:
Howdy, Sorry if this is a double post. First try seemed to go into hyperspace. I'm working on a personal project. It's going to be a multipart binary attachment downloader that will search alternate servers for missing pieces. This is the working code so far. It will walk a newsgroup and download and decode all the single part attachments. Just change server,user,password,group at the bottom to something less
10
2407
by: Julian Smith | last post by:
I've been playing with a function that creates an anonymous function by compiling a string parameter, and it seems to work pretty well: def fn( text): exec 'def foo' + text.strip() return foo This can be used like: def foo( x):
6
2174
by: Mark Miller | last post by:
I have a scheduled job that uses different XSL templates to transform XML and save it to disk. I am having problems with the code below. The problem shows up on both my development machine (Windows XP Pro SP 1, .Net Framework 1.1) and on our production server (Windows 2K SP 4, .Net Framework 1.1). I have simplified the code and data to isolate the problem. When I use the xsl:strip-space (Line 12) declaration in conjunction with the xsl:sort...
6
2597
by: rtilley | last post by:
s = ' qazwsx ' # How are these different? print s.strip() print str.strip(s) Do string objects all have the attribute strip()? If so, why is str.strip() needed? Really, I'm just curious... there's a lot don't fully understand :)
6
2360
by: eight02645999 | last post by:
hi can someone explain strip() for these : 'example' when i did this: 'abcd,words.words'
2
3640
by: =?iso-8859-1?b?cultaQ==?= | last post by:
Hi, I would like to rename files (jpg's ones) using a text file containing the new names... Below is the code that doesn't work : ***** #!/usr/bin/python #-*- coding: utf-8 -*- from os import listdir, getcwd, rename import re
3
2372
by: Colin J. Williams | last post by:
The Library Reference has strip( ) Return a copy of the string with the leading and trailing characters removed. The chars argument is a string specifying the set of characters to be removed. If omitted or None, the chars argument defaults to removing whitespace. The chars argument is not a
0
937
by: Kam-Hung Soh | last post by:
On Sun, 18 May 2008 00:47:55 +1000, Beema shafreen <beema.shafreen@gmail.comwrote: 1. Test with a small number of short files with a clear idea of the expected result. 2. Use better variable names. Names such as file1_search, file2_search, gi, gi2, A, B, C and D make it nearly impossible to understand your code.. --
6
4141
by: Christoph Zwerschke | last post by:
In Python programs, you will quite frequently find code like the following for removing a certain prefix from a string: if url.startswith('http://'): url = url Similarly for stripping suffixes: if filename.endswith('.html'): filename = filename
0
9811
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, weíll explore What is ONU, What Is Router, ONU & Routerís main usage, and What is the difference between ONU and Router. Letís take a closer look ! Part I. Meaning of...
0
9657
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
0
10822
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. Here is my compilation command: g++-12 -std=c++20 -Wnarrowing bit_field.cpp Here is the code in...
0
10532
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth. The Art of Business Website Design Your website is...
1
10577
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
0
10241
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
1
4443
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
2
4003
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.
3
3103
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.