Sqlite3 trouble

[Brontes]

I'm having some trouble with sqlite3 on my windows mobile device. Sometimes it happens that commit is not working.

Here is the code, that I'm using:

 
def OnSaveProfile(self, event):
        naziv = self.edt_title.text
        if not naziv:
            gui.Message.ok(u"Error!", u"Enter profile title!", icon='error', parent=self)
            return 0
        potnik = self.edt_st_potn.text
        if not potnik:
            gui.Message.ok(u"Error!", u"Enter number of worker!", icon='error', parent=self)
            return 0
        baza = self.com_baza.text
 
        #Here I connect to the database
        conn = orodja.povezava_na_bazo(self, "Nastavitve")
        c = conn.cursor()
        c.execute("Select nast_koda from Nastavitve where nast_koda = 'prof_%(naziv)s'" % {"naziv": naziv})
        dRez = c.fetchone()
        cins = conn.cursor()
        if dRez:#Checking if I need to update or insert new value
            retexec = cins.execute("Update Nastavitve set nast_vrednost = ? where nast_koda = ?", (potnik + "," + baza, "prof_"+naziv))
        else:
            retexec = cins.execute("Insert into Nastavitve Values (?, ?)", ("prof_"+naziv, potnik + "," + baza))
 
        print "Total changes", retexec.connection.total_changes #this returns 1 
 
        c2 = conn.cursor()
        c2.execute("Select nast_koda from Nastavitve where nast_koda = 'prof_%(naziv)s'" % {"naziv": naziv})
        dRez2 = c2.fetchone()
        #Test if new value is in the table before commit
        print "PredComm", dRez2#I always get the rasult here 
 
        ret = conn.commit()
        c1 = conn.cursor()
        c1.execute("Select nast_koda from Nastavitve where nast_koda = 'prof_%(naziv)s'" % {"naziv": naziv})
        dRez1 = c1.fetchone()
        #Test if new value is in the table after commit
        print "PoComm", dRez1#Sometimes happens that dRez1 == None
        print "Total changes", retexec.connection.total_changes #this returns 1 
 
        self.Osvezi_profile()
        pass
 

I call function OnSaveProfile when I click some button on form. This code is meant to insert or update some data in table 'Nastavitve'. It works most of time, but sometimes the data just doen't inserts into table. When I click the button again, then it works fine (sometimes not on first click, but on second, third, ...). There is always data in table before commit. After commit it is sometimes gone. It looks like that instead of commit sqlite3 executes rollback.

I've done integrity_check and it returned 'ok'. I have PRAGMA synchronous set to 2-FULL. I've tried different isolation_level-s with no success.

I'm having trouble only on windows mobile device. When I run the application on desktop PC it always works fine.

So is this a bug on Windows Mobile or am I doing something wrong?

Thanks in advance for answer.

[dwblas]

Generally, connect is used in following format
con = sqlite.connect(table_name)
Since we don't have the code for the orodja.povezava_na_bazo() function, there is no way to tell if the connection is established correctly.

[Brontes]

Here is the code of povezava_na_bazo:

def povezava_na_bazo(self, baza=""):
    import sqlite3
 
    #baza is the name of database. I need this bacause I use application with more than one database.
    if baza:
        baza = "_"+baza
    if os.name == 'ce':
        #this is used for mobile device
        conn = sqlite3.connect('\\Program Files\\Python25\\Baza\\ANA_Mob_prod%(baza)s.db' %{'baza':baza})
    else:    
        #this is used for desktop computer (testing)
        conn = sqlite3.connect('D:/Database/Mobilno/ANA_Mob_prod%(baza)s.db' %{'baza':baza})
    return conn
 

[dwblas]

All of the separate cursors may be confusing the database. The code should be more like:

        #Here I connect to the database
        conn = orodja.povezava_na_bazo(self, "Nastavitve")
        c = conn.cursor()
        c.execute("Select nast_koda from Nastavitve where nast_koda = 'prof_%(naziv)s'" % {"naziv": naziv})
        dRez = c.fetchone()
##        cins = conn.cursor()
        if dRez:#Checking if I need to update or insert new value
 
            ## use the same cursor
            retexec = c.execute("Update Nastavitve set nast_vrednost = ? where nast_koda = ?", (potnik + "," + baza, "prof_"+naziv))
        else:
            retexec = c.execute("Insert into Nastavitve Values (?, ?)", ("prof_"+naziv, potnik + "," + baza)) 

[Brontes]

Thanks for your answer. But unfortunately now is even worse. It happes more often, that the data doesn't saves.
I originally have the code written with only one cursor. But because of this trouble I tried with separate cursors.

Now I changed the code to:

    def OnSaveProfile(self, event):
        print "Start" 
        naziv = self.edt_title.text
        if not naziv:
            gui.Message.ok(u"Error!", u"Enter profile title!", icon='error', parent=self)
            return 0
        potnik = self.edt_st_potn.text
        if not potnik:
            gui.Message.ok(u"Error!", u"Enter number of worker!", icon='error', parent=self)
            return 0
        baza = self.com_baza.text
 
        #Here I connect to the database
        conn = orodja.povezava_na_bazo(self, "Nastavitve")
        c = conn.cursor()
        c.execute("Select nast_koda from Nastavitve where nast_koda = 'prof_%(naziv)s'" % {"naziv": naziv})
        dRez = c.fetchone()
        #cins = conn.cursor()
        if dRez:#Checking if I need to update or insert new value
            retexec = c.execute("Update Nastavitve set nast_vrednost = ? where nast_koda = ?", (potnik + "," + baza, "prof_"+naziv))
        else:
            retexec = c.execute("Insert into Nastavitve Values (?, ?)", ("prof_"+naziv, potnik + "," + baza))
 
        print "Total changes", retexec.connection.total_changes #this returns 1 
 
        #c = conn.cursor()
        c.execute("Select nast_koda from Nastavitve where nast_koda = 'prof_%(naziv)s'" % {"naziv": naziv})
        dRez2 = c.fetchone()
        #Test if new value is in the table before commit
        print "PredComm", dRez2#I always get the rasult here 
 
        ret = conn.commit()
        #c1 = conn.cursor()
        c.execute("Select nast_koda from Nastavitve where nast_koda = 'prof_%(naziv)s'" % {"naziv": naziv})
        dRez1 = c.fetchone()
        #Test if new value is in the table after commit
        print "PoComm", dRez1#Sometimes happens that dRez1 == None
        print "Total changes", retexec.connection.total_changes #this returns 1 
 
        print "End"
        conn.close()
        self.Osvezi_profile()
        pass
 
 

I added conn.close() at the end and it still doesn't make the difference.
This is what I got printed out (note prof_A98, prof_A94, prof_A93, prof_A92):
Start
Total changes 1
PredComm (u'prof_A99',)
PoComm (u'prof_A99',)
Total changes 1
End
Start
Total changes 1
PredComm (u'prof_A98',)
PoComm None
Total changes 1
End
Start
Total changes 1
PredComm (u'prof_A98',)
PoComm (u'prof_A98',)
Total changes 1
End
Start
Total changes 1
PredComm (u'prof_A97',)
PoComm (u'prof_A97',)
Total changes 1
End
Start
Total changes 1
PredComm (u'prof_A96',)
PoComm (u'prof_A96',)
Total changes 1
End
Start
Total changes 1
PredComm (u'prof_A95',)
PoComm (u'prof_A95',)
Total changes 1
End
Start
Total changes 1
PredComm (u'prof_A94',)
PoComm None
Total changes 1
End
Start
Total changes 1
PredComm (u'prof_A94',)
PoComm (u'prof_A94',)
Total changes 1
End
Start
Total changes 1
PredComm (u'prof_A93',)
PoComm None
Total changes 1
End
Start
Total changes 1
PredComm (u'prof_A93',)
PoComm (u'prof_A93',)
Total changes 1
End
Start
Total changes 1
PredComm (u'prof_A92',)
PoComm None
Total changes 1
End
Start
Total changes 1
PredComm (u'prof_A92',)
PoComm None
Total changes 1
End
Start
Total changes 1
PredComm (u'prof_A92',)
PoComm (u'prof_A92',)
Total changes 1
End
Start
Total changes 1
PredComm (u'prof_A91',)
PoComm (u'prof_A91',)
Total changes 1
End
Start
Total changes 1
PredComm (u'prof_A90',)
PoComm (u'prof_A90',)
Total changes 1
End

[dwblas]

Move line 32
ret = conn.commit()
to line 23, that is after the update/insert. I am still guessing, that doing a update, then select and fetchone, then committing, may be causing the problem. In theory that should not be a problem though. Also, inserts and updates are usually in the following format, which uses a dictionary not a "?". Try this as well. I am no SQLite expert, so you may have to comment everything except the errant code, and try updates one at a time until you find the data that has a problem, and try working backwards.

retexec = c.execute("Update Nastavitve set nast_vrednost = :dic_1 where nast_koda = :dic_2", {dic_1:potnik + "," + baza, dic_2:"prof_"+naziv})

[Brontes]

Now I tried this version:

        #Here I connect to the database
        conn = orodja.povezava_na_bazo(self, "Nastavitve")
        #conn.execute("begin immediate transaction")
        c = conn.cursor()
        c.execute("Select nast_koda from Nastavitve where nast_koda = 'prof_%(naziv)s'" % {"naziv": naziv})
        dRez = c.fetchone()
        #cins = conn.cursor()
 
        if dRez:#Checking if I need to update or insert new value
            sSelect = "Update Nastavitve set nast_vrednost = '%(vrednost)s' where nast_koda = '%(naziv)s'" % {"vrednost": potnik + "," + baza, "naziv": "prof_"+naziv}
        else:
            sSelect = "Insert into Nastavitve Values ('%(naziv)s', '%(vrednost)s') " % {"vrednost": potnik + "," + baza, "naziv": "prof_"+naziv}
        #Command execute on "conn" --> not on "c" (cursor)
        retexec = conn.execute(sSelect)
        conn.commit()
 

So without setting cursor for insert/update statement and commit just after insert/update. But it is still the same. On desktop PC works fine, but on Win Mobile device (or emulator) the data sometimes just doesn't inserts.

I also tried this version:

        conn = orodja.povezava_na_bazo(self, "Nastavitve")
        #conn.execute("begin immediate transaction")
        c = conn.cursor()
        c.execute("Select nast_koda from Nastavitve where nast_koda = 'prof_%(naziv)s'" % {"naziv": naziv})
        dRez = c.fetchone()
        #cins = conn.cursor()
 
        if dRez:#Checking if I need to update or insert new value
            sSelect = "Update Nastavitve set nast_vrednost = '%(vrednost)s' where nast_koda = '%(naziv)s'" % {"vrednost": potnik + "," + baza, "naziv": "prof_"+naziv}
        else:
            sSelect = "Insert into Nastavitve Values ('%(naziv)s', '%(vrednost)s') " % {"vrednost": potnik + "," + baza, "naziv": "prof_"+naziv}
        retexec = c.execute(sSelect)
        conn.commit()
 

So with setting cursor for insert/update statement and commit just after insert/update... Still the same...