473,320 Members | 2,189 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,320 software developers and data experts.

storing the data

Hi,

I have 13.8 GB of data.

The data is of the following:-

1: Proc Natl Acad Sci U S A. 2003 Aug 19;100(17):9768-73. Epub 2003 Jul 30.

Characterization of myotubularin-related protein 7 and its binding partner,
myotubularin-related protein 9.

Mochizuki Y, Majerus PW.

Washington University School of Medicine, Department of Internal Medicine, 660
S. Euclid Avenue, St. Louis, MO 63110, USA.

Myotubularin-related protein 7 (MTMR7) is a member of the myotubularin (MTM)
family. The cDNA encoding the mouse MTMR7 contains 1,983 bp, and the predicted
protein has a deduced molecular mass of 75.6 kDa. Northern and Western blot
analyses showed that MTMR7 is expressed mainly in brain and mouse neuroblastoma
N1E-115 cells. Recombinant MTMR7 dephosphorylated the D-3 position of
phosphatidylinositol 3-phosphate and inositol 1,3-bisphosphate [Ins(1,3)P2]. The
substrate specificity of MTMR7 is different than other MTM proteins in that this
enzyme prefers the water-soluble substrate. Immunofluorescence showed that MTMR7
is localized in Golgi-like granules and cytosol, and subcellular fractionation
showed both cytoplasmic and membrane localization of MTMR7 in N1E-115 cells. An
MTMR7-binding protein was found in an anti-MTMR7 immunoprecipitate from N1E-115
cells and identified as MTM-related protein 9 (MTMR9) by tandem mass
spectrometry. The coiled-coil domain of MTMR9 was sufficient for binding to
MTMR7. The binding of MTMR9 increased the Ins(1,3)P2 phosphatase activity of
MTMR7. Our results show that MTMR7 forms a complex with MTMR9 and
dephosphorylates phosphatidylinositol 3-phosphate and Ins(1,3)P2 in neuronal
cells.

Publication Types:
In Vitro
Research Support, U.S. Gov't, P.H.S.

PMID: 12890864 [PubMed - indexed for MEDLINE]


I have stored in files.

I want to store 13.8 GB of data in mysql database.

Is it reliable to store the data in database or in files?

The main concept here is user will specify some keyword say "cancer" it should

retrieve the data which contains the word cancer.(like mentioned above data will be retrieved).

will searching be faster if store in files or in database?

with regards
Archana
Nov 14 '07 #1
14 2636
heat84
118 100+
There is a trade off . Random access files may be faster but also remember that mySQL uses InnoDB and myISAM for search so you can also store the data in a database. The idea is avoiding sequential access as much as possible
Nov 14 '07 #2
pbmods
5,821 Expert 4TB
Heya, Archana.

MySQL is very good at this.
Nov 15 '07 #3
Heya, Archana.

MySQL is very good at this.

Hi,

Thanks for the link.

I went through the links.

I checked in my mysql database but it is giving an error.

error is this:-

ERROR 1064: You have an error in your SQL syntax near 'BOOLEAN MODE)' at line 1

I checked the mysql version and version is 3.23.58.

does full text searches will not work in mysql 3.23.58?

Is there any other way to be done?

since i have to use boolean expressions (for eg:-cancer and tissue) it should

retrieve the articles which has both these words i think i have to use boolean

mode.But that is giving an error.

what is compatible to mysql version 3.23.58.

How should i solve this problem?

with regards
Archana
Nov 15 '07 #4
pbmods
5,821 Expert 4TB
Heya, Archana.

MySQL 3.23.58 should be compatible with fulltext searching. I would strongly recommend that you upgrade (MySQL 3.23.58 was released over four years ago, with two major version releases since then). But if that's not an option, you should be able to get it working with a very minimal feature set.

What's the query look like?
Nov 15 '07 #5
Heya, Archana.

MySQL 3.23.58 should be compatible with fulltext searching. I would strongly recommend that you upgrade (MySQL 3.23.58 was released over four years ago, with two major version releases since then). But if that's not an option, you should be able to get it working with a very minimal feature set.

What's the query look like?

Hi,
The query which i have is:-

just i tried the example that was given.

Expand|Select|Wrap|Line Numbers
  1. CREATE TABLE articles (id INT UNSIGNED AUTO_INCREMENT NOT NULL PRIMARY KEY,title VARCHAR(200),body TEXT,FULLTEXT (title,body));
query OK..

Expand|Select|Wrap|Line Numbers
  1. INSERT INTO articles (title,body) VALUES ('MySQL Tutorial','DBMS stands for DataBase ...'),('How To Use MySQL Well','After you went through a ...'),('Optimizing MySQL','In this tutorial we will show ...'),('1001 MySQL Tricks','1. Never run mysqld as root. 2. ...'),('MySQL Security','When configured properly, MySQL ...');
it is inserting into the table.

If i give:-

Expand|Select|Wrap|Line Numbers
  1. mysql> select * from articles;                                                                                               +----+-----------------------+-------------------------------------+                                                         | id | title                 | body                                |
  2. +----+-----------------------+-------------------------------------+
  3. |  1 | MySQL Tutorial        | DBMS stands for DataBase ...        |
  4. |  2 | How To Use MySQL Well | After you went through a ...        |
  5. |  3 | Optimizing MySQL      | In this tutorial we will show ...   |
  6. |  4 | 1001 MySQL Tricks     | 1. Never run mysqld as root. 2. ... |
  7. |  5 | MySQL Security        | When configured properly, MySQL ... |
  8. +----+-----------------------+-------------------------------------+
  9. 5 rows in set (0.01 sec)
Expand|Select|Wrap|Line Numbers
  1. SELECT * FROM articles WHERE MATCH (title,body) AGAINST ('database' IN NATURAL LANGUAGE MODE); 
ERROR 1064: You have an error in your SQL syntax near 'NATURAL LANGUAGE MODE)' at line 1


It should retrieve the sentences where the database word is present.

I just tried with single word database to search and retrieve.

but it is giving an error.

usually my search string will be like this word1 AND word2 OR word3....

how do i proceed?

with regards
Archana
Nov 15 '07 #6
pbmods
5,821 Expert 4TB
Heya, Archana.

'IN NATURAL LANGUAGE MODE' is a feature specific to MySQL 5.1. Try removing it.

Here's a link to the fulltext reference for MySQL 3/4.

Note that boolean searching was not implemented until MySQL 4.
Nov 15 '07 #7
mwasif
802 Expert 512MB
Archanak kindly use proper CODE tags when posting. This makes the post more readable.
Nov 15 '07 #8
Heya, Archana.

'IN NATURAL LANGUAGE MODE' is a feature specific to MySQL 5.1. Try removing it.

Here's a link to the fulltext reference for MySQL 3/4.

Note that boolean searching was not implemented until MySQL 4.
Hi,

I tried doing but still it is giving error.

My query was this:-

Expand|Select|Wrap|Line Numbers
  1.  
  2. SELECT * FROM articles WHERE MATCH (title,body) AGAINST ('+MySQL' IN BOOLEAN MODE);
  3.  
  4.  
But i got an error..

ERROR 1064: You have an error in your SQL syntax near 'BOOLEAN MODE)' at line 1

How to solve?

version os 3.24
Dec 4 '07 #9
mwasif
802 Expert 512MB
BOOLEAN search was introduced in MySQL 4.0.1. You should use with BOOLEAN MODE.
Expand|Select|Wrap|Line Numbers
  1. SELECT * FROM articles WHERE MATCH (title,body) AGAINST ('MySQL');
Did you look at pbmods's provided link?
Dec 4 '07 #10
BOOLEAN search was introduced in MySQL 4.0.1. You should use with BOOLEAN MODE.
Expand|Select|Wrap|Line Numbers
  1. SELECT * FROM articles WHERE MATCH (title,body) AGAINST ('MySQL');
Did you look at pbmods's provided link?

Hi,

I just tried in mysql 5.0.22 but it is giving this error:-

ERROR 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'NATURAL LANGUAGE MODE)' at line 1

How to check this in the manual?

How to proceed?

with regards
Archana
Jan 2 '08 #11
pbmods
5,821 Expert 4TB
Heya, Archana.

NATURAL LANGUAGE MODE is a feature that was introduced in MySQL 5.1. You can omit this to get the same result in 5.0.
Jan 2 '08 #12
Heya, Archana.

NATURAL LANGUAGE MODE is a feature that was introduced in MySQL 5.1. You can omit this to get the same result in 5.0.
Hi,

I went through the manual of 3.23 in that they had given like this:-

MATCH (col1,col2,...) AGAINST (expr [search_modifier])

search_modifier: { IN BOOLEAN MODE | WITH QUERY EXPANSION }.

my query was like this :-

Expand|Select|Wrap|Line Numbers
  1.  SELECT * FROM data WHERE MATCH (title) AGAINST ('database' WITH QUERY EXPANSION);
  2.  
  3. SELECT * FROM data WHERE MATCH (title) AGAINST ('+MySQL' IN BOOLEAN MODE);
  4.  
But both gives same error.

You have an error in your SQL syntax near 'WITH QUERY EXPANSION)' at line 1.
You have an error in your SQL syntax near 'BOOLEAN MODE)' at line 1.

But fulltext search is supported by mysql 3.23 .I think boolean mode also i can't use.

with this version of mysql 3.23 can i perform searching?

with fulltext can i search multiple words with Parenthesized groups with boolean operators?

like (A AND (B OR C))?

I tried using only match operator :-

The rows are :-
Expand|Select|Wrap|Line Numbers
  1. mysql> select * from data;
  2. +-----------------------+
  3. | title                 |
  4. +-----------------------+
  5. | MySQL Tutorial        |
  6. | How To Use MySQL Well |
  7. +-----------------------+
  8. 2 rows in set (0.02 sec)
  9.  
  10.  SELECT * FROM data WHERE MATCH (title) AGAINST ('Tutorial');
  11.  
The result was EMPTY but the word is present in table.

How to go about it for multiple combinations with paranthesis?

with regards
Archana
Jan 11 '08 #13
pbmods
5,821 Expert 4TB
Heya, Archana.

Fulltext searching really only works when you have a relatively large data set. If you have too few records, you will not get results, even when it looks like you should.
Jan 12 '08 #14
Heya, Archana.

Fulltext searching really only works when you have a relatively large data set. If you have too few records, you will not get results, even when it looks like you should.
Hi,

I tried with "like" syntax it works.

Now i have one problem.

User enters a query term like this :- $query="(word1 AND word2) AND (word3 OR word4)";

In mysql i gave like this:-

Expand|Select|Wrap|Line Numbers
  1. select * from table_name where (name RLIKE '[[:<:]]word1[[:>:]]' AND name RLIKE '[[:<:]]word2[[:>:]]') AND (name RLIKE '[[:<:]]word3[[:>:]]' OR name RLIKE '[[:<:]]word4[[:>:]]');
  2.  
But in perl using $sth3=$dbh->prepare(query) i don't know how to form a query like above and give in prepare statement for fetching rows.

How to prepare a query statement to take the query terms given by user?

How to give in prepare statement like this :-

Expand|Select|Wrap|Line Numbers
  1. select * from table_name where ()?
  2.  
wht condition should i gave to take query terms?

If user gives only one term like "word1"

then i can give:-

Expand|Select|Wrap|Line Numbers
  1. "select * from table_name where name like '%$query%'";
  2.  
It fetches the rows..

But for above query combination "(word1 AND word2 ) combination how do i give?


I don't know to which forum this question belongs so i taught its more appropriate to Mysql so i posted here?

How to solve this?
Jan 15 '08 #15

Sign in to post your reply or Sign up for a free account.

Similar topics

2
by: Francisco | last post by:
I have this problem: I have a database with information about games, and users are able to vote for them. Everytime a user votes for a game I store the unique game name into a session variable (an...
1
by: Kay | last post by:
A linked list is storing several names. I want to make a queue if I input a name that is same as the linked list. How to make each node of a linked list storing a queue that are different with each...
7
by: Dave | last post by:
I have a system that basically stores a database within a database (I'm sure lots have you have done this before in some form or another). At the end of the day, I'm storing the actual data...
6
by: bissatch | last post by:
Hi, I am currently writing a news admin system. I would like to add the ability to add images to each article. What I have always done in the past is uploaded (using a form) the image to a...
1
by: VMI | last post by:
Is it possible to store the data in a datatable in the hard disk instead of the memory? By default, when a datatable's being filled, the table (and data) will remain in memory. Would it be possible...
6
by: Kyle Teague | last post by:
What would give better performance, serializing a multidimensional array and storing it in a single entry in a table or storing each element of the array in a separate table and associating the...
6
by: (PeteCresswell) | last post by:
User wants to go this route instead of storing pointers in the DB and the documents outside. Only time I tried it was with only MS Word docs - and that was a loooong time ago - and it seemed to...
20
by: Martin Jørgensen | last post by:
Hi, I'm reading a number of double values from a file. It's a 2D-array: 1 2 3 4 5 6 7 ------------- 1 3.2 2 0 2.1 3 9.3 4
7
by: Mike | last post by:
I have developed an application, for psyc patients.... they type in very personal information in a web form to help them work through problems in their lives. Once they enter the info, I encrypt...
4
by: Falcolas | last post by:
I personally feel that it's a bad idea. I've got to convince managerial types of this. Would some of you mind posting comments for and against the concept of storing data in XML attributes? If...
0
by: DolphinDB | last post by:
The formulas of 101 quantitative trading alphas used by WorldQuant were presented in the paper 101 Formulaic Alphas. However, some formulas are complex, leading to challenges in calculation. Take...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
by: ryjfgjl | last post by:
ExcelToDatabase: batch import excel into database automatically...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
1
by: PapaRatzi | last post by:
Hello, I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...
1
by: Shællîpôpï 09 | last post by:
If u are using a keypad phone, how do u turn on JavaScript, to access features like WhatsApp, Facebook, Instagram....
0
by: af34tf | last post by:
Hi Guys, I have a domain whose name is BytesLimited.com, and I want to sell it. Does anyone know about platforms that allow me to list my domain in auction for free. Thank you
0
by: Faith0G | last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.