473,411 Members | 1,975 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,411 software developers and data experts.

How to write to file using tail -f through a pipe to grep?

112 100+
Hi -- I'm looking to write to a file after piping output from tail -f through to grep:

#write to a file for all lines with "searchtext" within in error_log:
Expand|Select|Wrap|Line Numbers
  1. tail -f /var/error_log | grep searchtext > output.txt
  2.  
The above command fails even though the following command::
Expand|Select|Wrap|Line Numbers
  1. tail -f /var/error_log | grep searchtext
  2.  
does produce output, as expected. Nothing is getting through while redirecting to files after a tail -f... The following work:
Expand|Select|Wrap|Line Numbers
  1. tail -f /var/error_log > output.txt                //without grep
  2. tail /var/error_log | grep searchtext > output.txt   //without -f
  3.  
So it's just the combination of grep and tail with specifically the -f (following) option that causes the problem. It would be ideal to be able to do this with a single inline command. Any ideas?

thanks for the help!!
Feb 16 '11 #1

✓ answered by mac11

It's working, you just don't see anything for a while because of buffering. I think grep's the one doing the buffering, but I'm not sure.

I tested as follows (you need 3 commands going at once):

Producer:
Expand|Select|Wrap|Line Numbers
  1. while [ 1 ]; do date >> in.txt; done
Filter:
Expand|Select|Wrap|Line Numbers
  1. tail -f in.txt | grep Feb > out.txt
Show the results as they come out:
Expand|Select|Wrap|Line Numbers
  1. tail -f out.txt
If you run it like this you will see stuff coming out from the tail on out.txt (like expected), but if you put a sleep into the first command (slow up the producer) it will take quite a while for anything to show up in the tail of out.txt

Edit: I checked (man grep) and it's got the --line-buffered option, which will help you see output quicker

1 5241
mac11
256 100+
It's working, you just don't see anything for a while because of buffering. I think grep's the one doing the buffering, but I'm not sure.

I tested as follows (you need 3 commands going at once):

Producer:
Expand|Select|Wrap|Line Numbers
  1. while [ 1 ]; do date >> in.txt; done
Filter:
Expand|Select|Wrap|Line Numbers
  1. tail -f in.txt | grep Feb > out.txt
Show the results as they come out:
Expand|Select|Wrap|Line Numbers
  1. tail -f out.txt
If you run it like this you will see stuff coming out from the tail on out.txt (like expected), but if you put a sleep into the first command (slow up the producer) it will take quite a while for anything to show up in the tail of out.txt

Edit: I checked (man grep) and it's got the --line-buffered option, which will help you see output quicker
Feb 18 '11 #2

Sign in to post your reply or Sign up for a free account.

Similar topics

2
by: Thomas | last post by:
What's the quickest way to write and read 10.000 integer values ( or more ) to and from a file? Using struct somehow? The example in the docs shows how to handle to or three arguments, but is the...
3
by: Ben | last post by:
Hi all - I am having a bit of trouble and thought maybe someone in this group could shed some light. Here's the skinny... I am creating an automated process to import a bunch of text files...
10
by: Tony Stock | last post by:
Hi all, problem - need to read a log (text) file looking for certain entries, this file will still be written to by another 3rd party process, and hence constantly require monitoring (dare I say...
6
by: hpy_awad | last post by:
I am writing stings ((*cust).name),((*cust).address)to a file using fgets but rabish is being wrote to that file ? Look to my source please and help me finding the reason why this rabish is being...
16
by: ben beroukhim | last post by:
I have huge number of legacy code which use standard files functions. I would like to pass a memory pointer rather than a FILE pointer. I am trying to use FILEs in the code to refer to memory...
2
by: kelly | last post by:
Hi, I don't have a code to show you, what I need are references or algorithms so that I'm in a right track. By the way, thanks for introducing me the array or arrays. Now I can continue my...
3
by: JDeats | last post by:
I have some .NET 1.1 code that utilizes this technique for encrypting and decrypting a file. http://support.microsoft.com/kb/307010 In .NET 2.0 this approach is not fully supported (a .NET 2.0...
8
by: =?Utf-8?B?Q2hyaXMgRmluaw==?= | last post by:
I am trying to make a minor modification to the code below and need some assistance. Currently this code is using the java.util, java.util.zip, and java.io assemblies from the vjslib.dll assembly....
0
by: Arjun Sarankulu | last post by:
using System; using System.Collections.Generic; using System.ComponentModel; using System.Data; using System.Drawing; using System.Text; using System.Windows.Forms; using System.IO; ...
0
by: emmanuelkatto | last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud. Please let me know. Thanks! Emmanuel
0
BarryA
by: BarryA | last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
1
by: nemocccc | last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
0
by: Hystou | last post by:
There are some requirements for setting up RAID: 1. The motherboard and BIOS support RAID configuration. 2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...
0
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However,...
0
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can...
0
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven...
0
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows...
0
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.