473,322 Members | 1,538 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,322 software developers and data experts.

Fast String Processing in .NET

Hi,

Consider a server which receives delimiter based string data from client.

It needs to process this data very fast.

Now if its built in C++, one obvious idea would be to put the string on heap
as char* and then pass the pointer around, delimiter separated fields will
be accessed via pointers as well ( <map> of pointers to fields ). this will
mean that there is no extensive performance nor memory consumption.

So how this can be done in the most performance friendly way in .NET

Please let me know if there is more info req on the problem ( this is based
on a real life problem )

Thank you in advance
rawCoder
Jul 21 '05 #1
4 1523
Use your MSDN and Search for StringBuilder.

"rawCoder" <ra******@hotmail.com> wrote in message
news:er**************@tk2msftngp13.phx.gbl...
Hi,

Consider a server which receives delimiter based string data from client.

It needs to process this data very fast.

Now if its built in C++, one obvious idea would be to put the string on
heap
as char* and then pass the pointer around, delimiter separated fields will
be accessed via pointers as well ( <map> of pointers to fields ). this
will
mean that there is no extensive performance nor memory consumption.

So how this can be done in the most performance friendly way in .NET

Please let me know if there is more info req on the problem ( this is
based
on a real life problem )

Thank you in advance
rawCoder

Jul 21 '05 #2
StringBuilder doesnt exactly provide what i am looking for.
It does provide super fast concatenation and mutability but what I really
need is something like referencing substrings inplace.

rawCoder
Jul 21 '05 #3
AFAIK you could also access this a an array of chars, as a stream or even
using pointers...

You'll have likely to do some testing to select the appropriate method.

Patrice

--

"rawCoder" <ra******@hotmail.com> a écrit dans le message de
news:uZ**************@TK2MSFTNGP14.phx.gbl...
StringBuilder doesnt exactly provide what i am looking for.
It does provide super fast concatenation and mutability but what I really
need is something like referencing substrings inplace.

rawCoder

Jul 21 '05 #4
rawCoder wrote:
Consider a server which receives delimiter based string data from
client.

It needs to process this data very fast.
I always find such statements to be suspicious. This is the same as saying
"as fast as possible", which, given an infinite amount of development time,
could be quite fast. But no project will support an infinite amount of
development time.

If this is your requirement, change it to something concrete, like "must
process strings from sample data set X with an average time of 12 ms and a
maximum time of 50 ms" or something similar. If this is someone elses
requirement, get them to clarify it in a similar manner.

This allows you to determine when this requirement has been met and when you
can move on to other requirements such as scalability and robustness.
Now if its built in C++, one obvious idea would be to put the string
on heap as char* and then pass the pointer around, delimiter
separated fields will be accessed via pointers as well ( <map> of
pointers to fields ). this will mean that there is no extensive
performance nor memory consumption.

So how this can be done in the most performance friendly way in .NET


Well, for one thing, "in .NET" would include Managed C++ where you could do
this precisely as you've stated.

Secondly, presuming you want to use one of the libraries, I would start with
the Regex library. If you can define your text processing in terms of that,
you likely would find very good performance, given a precompiled RegEx
expression.

http://msdn.microsoft.com/library/de...classtopic.asp

(tiny version:)

http://tinyurl.com/9s84v
Thirdly, if that still isn't good enough, you may want to step back into
(Managed) C++ and look into using the Spirit parser.

HTH

--
Reginald Blue
"I have always wished that my computer would be as easy to use as my
telephone. My wish has come true. I no longer know how to use my
telephone."
- Bjarne Stroustrup (originator of C++) [quoted at the 2003
International Conference on Intelligent User Interfaces]
Jul 21 '05 #5

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

6
by: Cable | last post by:
Hello, I am hoping that someone can answer a question or two regarding file access. I have created an app that reads an image from a file then displays it (using OpenGL). It works well using...
6
by: G.Esmeijer | last post by:
Friends, I would like to read a text file (fixed length formaated) really fast and store the data into an Access database (2003). Using the streamreader and reading line by line, separating the...
4
by: rawCoder | last post by:
Hi, Consider a server which receives delimiter based string data from client. It needs to process this data very fast. Now if its built in C++, one obvious idea would be to put the string on...
10
by: Brent | last post by:
I'm quickly running out of available db connections with the code below. I like to think I'm closing everything as I go along, but I watch the connections shoot up as soon as the the method getIDs...
4
by: Alexis Gallagher | last post by:
(I tried to post this yesterday but I think my ISP ate it. Apologies if this is a double-post.) Is it possible to do very fast string processing in python? My bioinformatics application needs to...
4
by: Jean-François Blais | last post by:
Many must have asked that question. I intend to write a few programs which will be CPU intensive. I know a little of Java. Would a language that yields native code, instead of bytecode perfrom much...
15
by: Michael A. Covington | last post by:
Any thoughts about how to implement image processing algorithms to run fast in C#? Using GetPixel to access every pixel from a Bitmap seems to be rather time-consuming.
4
by: cyberdaemon | last post by:
Hi all, ok I have this problem where I need to process 50,000+ records from three different tables on two different servers thus 150,000+ records. Now I have got this program to work, however, it...
7
by: Karch | last post by:
I need to find the fastest way in terms of storage and searching to determine if a given string contains one of a member of a list of strings. So, think of it in terms of this: I have a string such...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...
1
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
1
by: Defcon1945 | last post by:
I'm trying to learn Python using Pycharm but import shutil doesn't work
1
by: Shællîpôpï 09 | last post by:
If u are using a keypad phone, how do u turn on JavaScript, to access features like WhatsApp, Facebook, Instagram....
0
by: af34tf | last post by:
Hi Guys, I have a domain whose name is BytesLimited.com, and I want to sell it. Does anyone know about platforms that allow me to list my domain in auction for free. Thank you
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome former...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.