473,248 Members | 1,465 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,248 software developers and data experts.

How to in Python

I've got a pointer to a position in a line of code that contains
either a digit or a period (decimal point). I've got this comment:

Numbers are one of these:
integers:
digit+
0xhex_digit+
decimals:
digit+.digit*[E['+'|'-']digit+]
.digit+[E['+'|'-']digit+]
digit+[.digit*]%
.digit+%

Common metacode: '*' = 0 or more, '+' = 1 or more, [] = optional, | =
or, ...

Now I need to instantiate the comment. How would an experienced Python
coder proceed?
Dec 19 '07 #1
8 1159
En Wed, 19 Dec 2007 14:02:00 -0300, <Ma************@gmail.comescribi�:
I've got a pointer to a position in a line of code that contains
either a digit or a period (decimal point). I've got this comment:

Numbers are one of these:
integers:
digit+
0xhex_digit+
decimals:
digit+.digit*[E['+'|'-']digit+]
.digit+[E['+'|'-']digit+]
digit+[.digit*]%
.digit+%

Common metacode: '*' = 0 or more, '+' = 1 or more, [] = optional, | =
or, ...

Now I need to instantiate the comment. How would an experienced Python
coder proceed?
Do you have to validate input based on that grammar? That is, do you want
a function like this?

def is_valid_number(input):
if ....
return True
return False

Or, do you want to consume characters starting from your pointer, stopping
when an invalid character appears?

This looks like one of those few cases where "Use a regular expression"
may be a good answer. If you like a more Pythonic approach, try using
Pyparsing <http://pyparsing.wikispaces.com/>

--
Gabriel Genellina

Dec 19 '07 #2
On Dec 20, 4:02 am, MartinRineh...@gmail.com wrote:
I've got a pointer to a position in a line of code that contains
either a digit or a period (decimal point). I've got this comment:

Numbers are one of these:
integers:
digit+
0xhex_digit+
decimals:
digit+.digit*[E['+'|'-']digit+]
.digit+[E['+'|'-']digit+]
digit+[.digit*]%
.digit+%

Common metacode: '*' = 0 or more, '+' = 1 or more, [] = optional, | =
or, ...

Now I need to instantiate the comment. How would an experienced Python
coder proceed?
Use a proper lexer written by somebody who knows what they are doing,
as has already been recommended to you.
Dec 19 '07 #3


John Machin wrote:
Use a proper lexer written by somebody who knows what they are doing,
as has already been recommended to you.
My lexer returns a MALFORMED_NUMBER token on '0x' or '0x '. Try that
in Python.

Dec 21 '07 #4
If I get to add multi-line strings today, I'll have a complete
tokenizer. Interior looks a lot like C minus semi-colons. (Though I
did figure out that there wasn't any need for tokens that didn't come
from a real to have a doubleValue field. In C++ or Java all the Tokens
had a doubleValue, because one needed it.)

Theory: there are some problems which, by their nature, end up looking
a lot like C, regardless of language.

I wrote a Perl-to-HTML pretty-printer. It lets you embed HTML in
comments and creates a table of contents hyperlinked to the individual
functions. It's at http://www.MartinRinehart.com , output and source
code, in the Articles section.

I started with a Perl-style solution: regex for the chars left of "#"
and for the chars to the right. One line of Perl. Nice, except that it
won't work when applied to itself. It will split the line based on the
"#" in the regex, of course. Kludged around that, but then met "#"
embedded in strings, "#" embedded in regex passed to functions, ...

Ended up marching down the input line, one character at a time, like a
C program.
Dec 21 '07 #5
On Dec 21, 2007 7:25 AM, <Ma************@gmail.comwrote:
>

John Machin wrote:
Use a proper lexer written by somebody who knows what they are doing,
as has already been recommended to you.

My lexer returns a MALFORMED_NUMBER token on '0x' or '0x '. Try that
in Python.
Is there some reason that you think Python is incapable of
implementing lexers that do this, just because Python lexer accepts
it? Note that if you're using your lexer to mark up or pretty print or
whatever Python source, it's wrong - 0x is (rightly or not) a valid
Python literal.
Dec 21 '07 #6


Chris Mellon wrote:
Is there some reason that you think Python is incapable of
implementing lexers that do this, just because Python lexer accepts
it?
Absolutely not. My opinion is that it's a bug. A very, very minor bug,
but still six-legged.
Note that if you're using your lexer to mark up or pretty print or
whatever Python source, it's wrong - 0x is (rightly or not) a valid
Python literal.
My lexer is for my language, Decaf, which, in this particular, is the
same as Python. Here's what I find at at python.org/ref: (2.4.4).

hexinteger ::= "0" ("x" | "X") hexdigit+

Implementation differs from specification. In this case, I think the
spec is more sensible.
Dec 21 '07 #7
On Dec 21, 2007 1:02 PM, <Ma************@gmail.comwrote:
>

Chris Mellon wrote:
Is there some reason that you think Python is incapable of
implementing lexers that do this, just because Python lexer accepts
it?

Absolutely not. My opinion is that it's a bug. A very, very minor bug,
but still six-legged.
I understand the argument but I'm wondering why you're asking
questions on this list about it. You don't seem to be implementing the
lexer in Python (because otherwise the question wouldn't have come
up), and you don't seem to be parsing Python code. Where's the python
angle? Not that this list is totally intolerant of offtopic
discussions, but you don't seem to even have a hint of Python interest
here - the questions seem to be better suited for comp.parsers.general
or something.
Note that if you're using your lexer to mark up or pretty print or
whatever Python source, it's wrong - 0x is (rightly or not) a valid
Python literal.

My lexer is for my language, Decaf, which, in this particular, is the
same as Python. Here's what I find at at python.org/ref: (2.4.4).

hexinteger ::= "0" ("x" | "X") hexdigit+

Implementation differs from specification. In this case, I think the
spec is more sensible.
I tend to consider "what the parser actually accepts" rather than
"what the grammar specifies" to be normative when writing pretty
printers or syntax highlighters.
Dec 21 '07 #8


Chris Mellon wrote:
You don't seem to be implementing the
lexer in Python
I am absolutely implementing my language in Python, a language I have
now been writing for two entire weeks. This list has been more than
helpful, tolerating numerous newbie questions.
Dec 22 '07 #9

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

0
by: abbasky | last post by:
### Vandf component communication method one: data sharing ​ Vandf components can achieve data exchange through data sharing, state sharing, events, and other methods. Vandf's data exchange method...
0
Git
by: egorbl4 | last post by:
Скачал я git, хотел начать настройку, а там вылезло вот это Что это? Что мне с этим делать? ...
0
by: MeoLessi9 | last post by:
I have VirtualBox installed on Windows 11 and now I would like to install Kali on a virtual machine. However, on the official website, I see two options: "Installer images" and "Virtual machines"....
0
by: DolphinDB | last post by:
The formulas of 101 quantitative trading alphas used by WorldQuant were presented in the paper 101 Formulaic Alphas. However, some formulas are complex, leading to challenges in calculation. Take...
0
by: Aftab Ahmad | last post by:
Hello Experts! I have written a code in MS Access for a cmd called "WhatsApp Message" to open WhatsApp using that very code but the problem is that it gives a popup message everytime I clicked on...
0
by: ryjfgjl | last post by:
ExcelToDatabase: batch import excel into database automatically...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: marcoviolo | last post by:
Dear all, I would like to implement on my worksheet an vlookup dynamic , that consider a change of pivot excel via win32com, from an external excel (without open it) and save the new file into a...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.