Identifying mismatched sets with two non-unique criteria

Joel Marion

First, my apologies if the title of this post is overly cryptic. I have a data set identifying events (academic courses) including start and end times. I want to verify that the times entered fit into a prescribed set of start and end times (slots), and spit out a report flagging courses that do not fit the prescribed slots. I also need to exclude courses with Section and/or Location codes that don't need to be checked.

I have the following tables:
Output: Course, Sec, Start, End, Location
Slots: Slot, SlotStart, SlotEnd, Term
Roomless (location codes): ID, OffcLoc
RoomlessSections: id, Section

My process is:
1. exclude Output.Sec = RoomlessSections.Section
2. exclude Output.Location = Roomless.OffcLoc
3. identify Slots.Slot where Output.Start = Slots.SlotStart
4. identify Slots.Slot where Output.End = Slots.SlotEnd
5. produce a report with the following:

a. Start not found in Slots
b. End not found in Slots
c. Start and End not in matching Slot

Further complicating things, the Slot Start and End times are not unique values (For example: Slot 6 is 11:30-12:20 and Slot 9 is 11:30-12:45), so I have to compare a list of possible slots based on each of Start and End to see if there is a match.

My approach has been to first try to produce the results in separate queries, thus:

Possible Start Slots:
and Possible End Slots:
Then compare the two:
Finally, I use a fourth Query to analyze the data:
I'm using a make table so that I can run the report separate from the query, as (I realize) running four queries like this is extremely inefficient and slow.

I know I'm going about this the hard way, but I don't know how else to build this kind of comparison.

Extra Info :
My Slots table looks like this:
Notice how a number of start and end times repeat, and not necessarily in the same pairings. ie: 2:30 pm SlotStart shows up three times, with two different End times.

So, if I have a course with a 2:30 start, I need to be able to accept 3:45 pm or 5:15 pm as valid End times. This is why I'm trying to identify along the lines of 'if a 2:30 start could possibly be slots 11, 13 or 15, does my end time match the end times for these slots.'

So far, my efforts don't seem to be working. I think my problem is in my JOINs, but I'm not familiar enough with this kind of query to understand why.

Jan 31 '12 #1

ADezii

This can probably be attempted in Code, if you like I'll give it a shot when I hqqve some time.

Feb 2 '12 #2

Joel Marion

I'd love the help. If you can even suggest the type of method you might use I can start looking. I'm just so new to VBA I don't know where to start.

Feb 2 '12 #3

ADezii

In general, the following Code will:

1. Accept Input from the User for the Start and End Times. This is accomplished via 2 Text Boxes (txtStartTime and txtEndTime) on a Form.
2. Check the Start Time entered with existing Values in the Slots Table.
3. If the Start Time does not exist in Slots, notify the User then Exit.
4. If the Start Time exists in Slots, check the entered End Time against End Times in Slots for the given Start Time.
5. If the End Time does not exist (Start Time does), notify User and Exit.
6. If both the Start and End Times are Valid Entries, post the available Slots (1 to 15) in a List Box (lstSlots) on the Form.
7. Realizing that all this may be totally confusing to you, in addition to the posted Code, I've also Attached a Demo that I created for you.
8. Study the Demo carefully, and should you have any questions, feel free to ask.

P.S. - I see no way how this can be accomplished via SQL, but I am not the person to ask regarding this. Hopefully, NeoPa, Rabbit, or one of the other SQL Phenoms will see this Thread and give their opinion.

Feb 2 '12 #4

Rabbit

Basically you want to find
But do you even need c? If they're in c then they must be in either a or b.

Feb 2 '12 #5

Joel Marion

Rabbit: yes, c is where I check whether or not courses that are found in both start and end slots have a matching slot number (if they do, I want to exclude them from my report).

ADezii: I haven't had a chance to take a look at your example, but thank you for providing it!

Feb 2 '12 #6

Rabbit

What I'm trying to say is there's no need to check for c. Because if they fall under c, they must necessarily fall under a or b. The query should be pretty close to what you're looking for.

Feb 2 '12 #7

NeoPa

I imagine that should probably read :

a. [Start] but not [End] found in Slots.
b. [End] but not [Start] found in Slots.
c. Neither [Start] nor [End] found in Slots

This includes all the same records, but simply separates them in the report into different batches.

If so, then your concept is the approach that I would have recommended. It has the beauty of clearing away duplicates automatically from the result set, which is inherent in the UNION clause.

What it won't do as it stands though, is indicate the status values required for the report, as only the [Output] data is included in the SELECT clause. It's certainly a very important basis from which to start though.

Feb 2 '12 #8

Joel Marion

Rabbit, you're right that if something falls under a. and b. it would necessarily fall under c., making c. redundant. But the c. I'm looking for is whether or not, in cases when [Start] and [End] are found in Slots, whether or not the slots (or possible slots) for each match.

Thus:
a. [Start] but not [End] found in Slots ("End time mismatch")
b. [End] but not [Start] found in Slots ("Start time mismatch")
c. [Start] and [End] found in Slots, but Slots don't match

Option C example (matching slots):
[Start]= 2:30 PM
[End] = 3:45 PM
I need to figure out that [Start] can be either slot 11 or 13, and I need to compare this to [End] which can be (in this example) either slot 11 or 13. This example would indicate a 'normal' course that doesn't need to be flagged.

In some cases there will be multiple matching slots (as in the above example), in some cases only one slot option will match, and in other cases no slot option will match.

Option C example 2 (non-matching slots):
[Start] = 1:00 PM
[End] = 3:45 PM
In this case, [Start] is slot 10 and [End] could be either slot 11 or 13. Because the possible slots don't match, I want to flag this course as being outside of the normal slot system.

The code offered by NeoPa in the previous comment provides for Options a. and b. but naturally excludes c.

I think I'm going to need to run a separate query to do the slot matching. How would I identify the possible slots in order to compare them? This is a bit of a shot in the dark, but could I store the options in an array in a DO loop?

Feb 2 '12 #9

Rabbit

As long as it doesn't matter what the actual error classification is, you just need to add an additional union query to my original suggestion.

Feb 2 '12 #10

NeoPa

I'm starting to realise this concept won't work as intended :-( I know Rabbit will correct me if I go astray, but here's my latest thinking.

It seems that the original question was mis-expressed, as point 5c (Originally a second 5a, but that's fixed as I just noticed it now.) clearly stated that records where neither [Start] nor [End] were required, whereas post #6 indicates categorically they now are not. Not that this was the reason for the earlier SQL being wrong at all, at least not as far as I'm concerned, but it does effect the SQL we're now looking for of course.

To make life simpler I'll create SQL sets that can be saved as QueryDefs in the database. This can all be done with subqueries in SQL etc (See Subqueries in SQL.) but it can lead to some very complicated (verbose) SQL with many repetitions (especially using Jet SQL as is native in Access).

qryOutputNoRooms :
qryOutputMatch :
qryResults :
I should clarify that tSS.SlotStart and tSE.SlotEnd are there to be used in the report to indicate what type of problem has been identified. A Null in either position indicates the missing match.

Feb 2 '12 #11

Rabbit

You know, I think it's been overcomplicated. Isn't the actual goal to identify everything without an exact match? Isn't this all that's needed?

Yes, this doesn't identify the "error" but I don't think that was needed in the first place. All that was needed was to identify the records.

Feb 2 '12 #12

NeoPa

Having just read Joel's latest explanation of point 5c in post #9 I was further confused, so I went back over post #6 to realise I'd been confused by the explanation there last time. I think I eventually understand what he was trying to say (which was closer to my original understanding of it before the explanation) and this means that the solution I provided in post #11 has three extraneous lines at the end. Essentially, lines #16 through #18 are not necessary for this scenario. The SQL finishes neatly at line #15 and if situation 5c occurs then neither tSS.SlotStart nor tSE.SlotEnd will be Null.

Feb 2 '12 #13

NeoPa

I don't believe so. That doesn't provide information with which to indicate which of the three scenarios (5a, 5b or 5c) each record falls into.
I'm not sure why you'd say that. Point #5 seems to specify the requirement to identify which type each failed record falls into.

Feb 2 '12 #14

Rabbit

I would contend that this information was not needed in the first place. I think the OP was only identifying the types of records they want to see rather than indicating that it also be part of the result set.

P.S. Nevermind, I was wrong. I took a look at the original post's fourth query and the type of error was in there.

Feb 2 '12 #15

NeoPa

At last!

Not that I feel any form of schadenfreude. Simply that I was starting to feel that every time we disagreed I turned out to be the one having to admit I was wrong. You still win on points of course, but at least it's no longer a whitewash :-D

Feb 2 '12 #16

Rabbit

I think neither of our queries work. Because of the hitherto unforeseen 5d error. Where neither start nor end is in the table.

Feb 2 '12 #17

NeoPa

I believe the SQL in post #11 (excluding lines #16 through #18 as amended in post #13) handles that OK. If you're convinced otherwise I'll have to knock up a test-rig to test it out, but I thought that was handled automatically in my code.

Feb 2 '12 #18

Rabbit

I would have try some sample data but I don't think your solution would identify c or d errors. I believe the errors are defined as:

a. start is in one of the records in the table but end is not in any of the records.
b. end is in one of the records in the table but start is not in any of the records.
c. either start or end is in one of the records in the table but the other is not in any of the records.
d. (new) start and end does not exist is any of the records in the table.

Feb 3 '12 #19

NeoPa

It appears there's still some confusion over the specification as described in point #5. Let me see if I can clarify things a little :

Possible Scenarios :

1. A matching [Slots] record is found where Output.Start = Slots.SlotStart and Output.End = Slots.SlotEnd.
2. [Slots] record(s) are found that match Output.Start = Slots.SlotStart but not Output.End = Slots.SlotEnd.
3. [Slots] record(s) are found that match Output.End = Slots.SlotEnd but not Output.Start = Slots.SlotStart.
4. There are [Slots] records that match Output.Start = Slots.SlotStart and Output.End = Slots.SlotEnd, but none that do both.
5. Neither Output.Start nor Output.End have matching values anywhere in [Slots].

My understanding is that all records are required from [Output] except those in scenario #1.

As such, I believe your scenario d was, indeed, previously unspecified, and matches scenario #5 above. However, your scenario c describes a set consisting of scenarios #2 & #3 above, rather than scenario #4.

All that being said, I would expect my code, as it works on the simple basis of excluding scenario #1 records, to cover all the other scenarios adequately (unless I have a flaw in my logic somewhere).

The values contained in Slots.SlotStart and Slots.SlotEnd will indicate which scenario a particular record belongs to as follows :

1. This never occurs due to line #15 in SQL.
2. Slots.SlotStart Is Not Null but Slots.SlotEnd Is Null.
3. Slots.SlotStart Is Null but Slots.SlotEnd Is Not Null.
4. Neither of Slots.SlotStart nor Slots.SlotEnd is Null.
5. Both of Slots.SlotStart and Slots.SlotEnd are Null.

Feb 3 '12 #20

NeoPa

As the last three lines of that SQL may still be muddying the waters for people, and also because, now UNION is no longer included in the mix there may be duplication of records in my previous version, I will post an amended version which includes the DISTINCT predicate to handle that likelihood (It also saves having to scroll down so far to check the details) :

qryResults :
[qryOutputNoRooms] and [qryOutputMatch] may still be found in their original versions in post #11.

PS. I also found a mis-reference in the original to tO.* in the SELECT line which has now been fixed in the original.

Feb 3 '12 #21

Rabbit

My bad, you are absolutely right. I just had to work it out on paper to see it.

Feb 3 '12 #22

NeoPa

: - )

Feb 3 '12 #23

Joel Marion

I've been trying to post since Friday, and the Bytes server keeps timing out on me, so here goes again:

To recap: I'm trying to identify
a. [Start] but not [End] found in Slots ("End is out of slot")
b. [End] but not [Start] found in Slots ("Start is out of slot")
c. [Start] and [End] are found in Slots, but Slots don't match ("cross-slot")

Which I've tried to break down into some logic:

qrySlotCheck:
qryMatched:
(note that I renamed qOWR.ID here because I was getting an error in my main query due to ID showing up twice.)

and qryOutputWithRooms:
I think this is giving me the results I need, but I have to spend some time with the results to be sure.

I do have two major concerns with this code:

1. qryOutputWithRooms produces 1607 records, while qrySlotCheck produced 1591 records. I need to compare the records to see which ones were dropped, but it gets tricky due to the second issue:

2. Running qrySlotCheck took nearly half an hour! I think I'm approaching some record for inefficient code. I'm sure this is because I'm running huge sub-queries against each record, but I'm still working on finding another way to produce the results.

I hope the logic in qrySlotCheck clears up some of the confusion about what it is I'm looking for. Specifically, I need to be able to produce explicit comments regarding which of the three issues noted above is being flagged.

I did consider trying some kind of flagging in a UNION, like
Even if I used this as the first half of a query that then went on to check matched slots, this doesn't allow for the specific reporting I'm looking for (particularly where both Start and End are not found).

Would it be better to run the whole thing in VBA, turn my sub-queries into make tables at the beginning of the process, and delete them when I'm done with them? I would think this would at least speed things up.

Feb 6 '12 #24

Joel Marion

I decided to run with the VBA idea, and built my SQL into some VBA, using make tables instead of queries. This sped things up considerably.

Here's what I came up with:

Feb 6 '12 #25