473,320 Members | 1,947 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,320 software developers and data experts.

Laundry list

I apologize for the long background story, but my project is a tad esoteric.

Technical data on individual stars are difficult to find, and what searchable databases I've found are geared more towards astronomers than lay people. At the same time, large catalogues of stars are freely available on the web as text files.

My project involves two scripts. The first script will mine the data out of these text files, generate some derived data like a 3D Cartesian coordinate (x, y, z), and combine it all into one (probably CSV) text file. The second script, naturally, will be a script to search the combined file based on user-provided criteria. I will also be setting up several presets for search criteria, such as for particularly Sun-like stars. The catch is that my programming skills are very rudimentary.

At the moment, what I have is a script that converts one of the files into CSV (the original format being character columns, e.g. column 1--4 is the catalog ID number, column 5--10 the type, etc.), a second script that does the mining and generation of the "combined file", and a script that lets the user input the ID number of a star, a distance, and "1" if they want to filter for Sun-like stars, and generates an unsorted file with all (or just Sun-like) stars within the specified distance of the target.

One big gap in my knowledge is efficient search methods, both for the search script itself and when building the combined file (checking to see if a given star is already in the combined file). Another is how to insert a new line to a file without having to rewrite the entire file (also for adding additional catalogue data to the combined file). A third, somewhat related, gap is how to sort the output from searches. Finally, I suspect there's a way to process the original files directly, without having to convert them all to CSV first (the main problem being how to handle the whitespace used to make the columns neat).

I've done some googling, but haven't found anything online (I recently moved to Argentina, so I can't just go out and buy a book), so I'm asking as much for pointers to tutorials and instructions as I am direct suggestions on how to do things.

Thanks in advance for your help.
Jan 12 '08 #1
2 1055
eWish
971 Expert 512MB
Have you considered using a database for this? It would be you most efficient and practical solution. Using a database would make searching simple. You would just need to parse the text files to get the data you want to store. Then insert it into a well structured database.

--Kevin
Jan 12 '08 #2
KevinADC
4,059 Expert 2GB
Really, I have to agree. What you want to learn is SQL, and probably MySQL in specific. Then if you want to use perl as the interface between the user and the database you will use the DBI module along with the appropriate databse driver, DBD::mysql for example.

But, it could be done with text files if you really want to go that route.
Jan 12 '08 #3

Sign in to post your reply or Sign up for a free account.

Similar topics

6
by: massimo | last post by:
Hey, I wrote this program which should take the numbers entered and sort them out. It doesn¹t matter what order, if decreasing or increasing. I guess I'm confused in the sorting part. Anyone...
10
by: Kent | last post by:
Hi! I want to store data (of enemys in a game) as a linked list, each node will look something like the following: struct node { double x,y; // x and y position coordinates struct enemy...
24
by: Robin Cole | last post by:
I'd like a code review if anyone has the time. The code implements a basic skip list library for generic use. I use the following header for debug macros: /* public.h - Public declarations and...
4
by: JS | last post by:
I have a file called test.c. There I create a pointer to a pcb struct: struct pcb {   void *(*start_routine) (void *);   void *arg;   jmp_buf state;   int    stack; }; ...
3
by: chellappa | last post by:
hi this simple sorting , but it not running...please correect error for sorting using pointer or linked list sorting , i did value sorting in linkedlist please correct error #include<stdio.h>...
0
by: drewy2k12 | last post by:
Heres the story, I have to create a doubly linked list for class, and i have no clue on how to do it, i can barely create a single linked list. It has to have both a head and a tail pointer, and...
10
by: AZRebelCowgirl73 | last post by:
This is what I have so far: My program! import java.util.*; import java.lang.*; import java.io.*; import ch06.lists.*; public class UIandDB {
0
by: Atos | last post by:
SINGLE-LINKED LIST Let's start with the simplest kind of linked list : the single-linked list which only has one link per node. That node except from the data it contains, which might be...
12
by: kalyan | last post by:
Hi, I am using Linux + SysV Shared memory (sorry, but my question is all about offset + pointers and not about linux/IPC) and hence use offset's instead on pointers to store the linked list in...
0
by: saijin | last post by:
I'm planning to call a list of data from an XML file but when I duplicate the content inside the <data></data> it is not showing anything Here's the ActionScript 3.0 import...
0
by: DolphinDB | last post by:
The formulas of 101 quantitative trading alphas used by WorldQuant were presented in the paper 101 Formulaic Alphas. However, some formulas are complex, leading to challenges in calculation. Take...
0
by: ryjfgjl | last post by:
ExcelToDatabase: batch import excel into database automatically...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
0
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
0
by: Defcon1945 | last post by:
I'm trying to learn Python using Pycharm but import shutil doesn't work
0
by: Shællîpôpï 09 | last post by:
If u are using a keypad phone, how do u turn on JavaScript, to access features like WhatsApp, Facebook, Instagram....
0
by: Faith0G | last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.