I've got a way for the user to delete the record. The user just didn't know it. I might end up putting a Delete button next to the Undo button so that they can find it easier.
An idea just occurred to me. If a user starts typing a new record and decides they don't want a new record, is it possible to delete the record before it is saved? If so, then I could just replace my Undo button with a Delete button that will work as both my Undo and Delete button. I know the functions are totally different, but in this case, the Undo wasn't to undo changes to an existing record, but to stop the creation of a new record if the user found out that the order had already been inputted.