422,530 Members | 1,031 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 422,530 IT Pros & Developers. It's quick & easy.

Shell run-time error 5, invalid procedure call or argument

P: 17
ok, shell always used to be easy for me, now its starting to get annoying cause i dont know wats wrong

heres the simplist code possible:

Expand|Select|Wrap|Line Numbers
  1. Private Sub IExplorer_Click()
  2. a = Shell("C:\Program Files\Internet Explorer\Iexplore.exe http://www.google.ca", vbMaximizedFocus)
  3. End Sub
i have a program thats been on my desktop for about a month now that uses these 3 lines of code(supposed to be 3 lines but it doesnt quite fit) and its been working perfectly up till today. I attempted to run it, then when it got to this part, it highlighted

Expand|Select|Wrap|Line Numbers
  1. a = Shell("C:\Program Files\Internet Explorer\Iexplore.exe http://www.google.ca", vbMaximizedFocus)
and a messagebox appeared saying run-time error 5, invalid procedure call or argument. I tried making an even simpler program that just opened notepad during form_load, but it still didnt work, same error. It doesnt make any sence, all that stuff used to work perfectly.
Aug 20 '07 #1
Share this Question
Share on Google+
24 Replies


kadghar
Expert 100+
P: 1,295
ok, shell always used to be easy for me, now its starting to get annoying cause i dont know wats wrong

heres the simplist code possible:

Expand|Select|Wrap|Line Numbers
  1. Private Sub IExplorer_Click()
  2. a = Shell("C:\Program Files\Internet Explorer\Iexplore.exe http://www.google.ca", vbMaximizedFocus)
  3. End Sub
i have a program thats been on my desktop for about a month now that uses these 3 lines of code(supposed to be 3 lines but it doesnt quite fit) and its been working perfectly up till today. I attempted to run it, then when it got to this part, it highlighted

Expand|Select|Wrap|Line Numbers
  1. a = Shell("C:\Program Files\Internet Explorer\Iexplore.exe http://www.google.ca", vbMaximizedFocus)
and a messagebox appeared saying run-time error 5, invalid procedure call or argument. I tried making an even simpler program that just opened notepad during form_load, but it still didnt work, same error. It doesnt make any sence, all that stuff used to work perfectly.
Hi,
I think this is because of the second argument, "vbmaximizedfocus" maybe you don have the constants set, why dont you try using the constan's value instead, something like

a = Shell("C:\Program Files\Internet Explorer\Iexplore.exe http://www.google.ca", 3)

hope that helps
Aug 20 '07 #2

P: 17
Hi,
I think this is because of the second argument, "vbmaximizedfocus" maybe you don have the constants set, why dont you try using the constan's value instead, something like

a = Shell("C:\Program Files\Internet Explorer\Iexplore.exe http://www.google.ca", 3)

hope that helps
nope, didnt work = ( , thanks for making an attempt tho
Aug 21 '07 #3

Expert 5K+
P: 8,435
I think this is because of the second argument, "vbmaximizedfocus" maybe you don have the constants set...
That would produce a compile error, not runtime.

I thought somebody was saying recently that Shell couldn't handle a path with spaces in it. Perhaps ShellExecute is better. Have a look at this thread.
Aug 21 '07 #4

pureenhanoi
100+
P: 175
ok, shell always used to be easy for me, now its starting to get annoying cause i dont know wats wrong

heres the simplist code possible:

Expand|Select|Wrap|Line Numbers
  1. Private Sub IExplorer_Click()
  2. a = Shell("C:\Program Files\Internet Explorer\Iexplore.exe http://www.google.ca", vbMaximizedFocus)
  3. End Sub
i have a program thats been on my desktop for about a month now that uses these 3 lines of code(supposed to be 3 lines but it doesnt quite fit) and its been working perfectly up till today. I attempted to run it, then when it got to this part, it highlighted

Expand|Select|Wrap|Line Numbers
  1. a = Shell("C:\Program Files\Internet Explorer\Iexplore.exe http://www.google.ca", vbMaximizedFocus)
and a messagebox appeared saying run-time error 5, invalid procedure call or argument. I tried making an even simpler program that just opened notepad during form_load, but it still didnt work, same error. It doesnt make any sence, all that stuff used to work perfectly.
I sometime meet this type of error. (Worked-code goes fault). I tried resolving problem and found the reason: missing one or more library in preference.
On the Project menu, choose Preference. One dialoge appears show all preferences of your project. If you see atleast one line have "...Missing...library...", so it is the fault.
If this library doesnot need for project, so simply un-check the library and click OK. If this really need, you can see the path to the ".dll" file. If the file is missing, u need copy one file to that location, or maybe install some other soft-wares that provide this "dll" file.
Aug 21 '07 #5

Expert 5K+
P: 8,435
I sometime meet this type of error. (Worked-code goes fault). I tried resolving problem and found the reason: missing one or more library in preference...
Are you sure you don't mean reference?
Aug 21 '07 #6

P: 17
That would produce a compile error, not runtime.

I thought somebody was saying recently that Shell couldn't handle a path with spaces in it. Perhaps ShellExecute is better. Have a look at this thread.
I've heard that too, and I also tried ShellExecute cause of it, but it still wasn't working, and besides, it doesn't make any sense that Shell can't handle spaces cause it was working before ...

I sometime meet this type of error. (Worked-code goes fault). I tried resolving problem and found the reason: missing one or more library in preference.
On the Project menu, choose Preference. One dialoge appears show all preferences of your project. If you see atleast one line have "...Missing...library...", so it is the fault.
If this library doesnot need for project, so simply un-check the library and click OK. If this really need, you can see the path to the ".dll" file. If the file is missing, u need copy one file to that location, or maybe install some other soft-wares that provide this "dll" file.
Ok, I tried that, but I didn't find anything saying missing library, and there's only 4 things checked, so as of now, I'm gonna try searching on Google for the dll file that shell uses, if I can't find anything that way, then I'll just select all the references and test if it works, if it doesn't, then I'm stuck again, if it does work, then I have to find out which one it is lmao...

EDIT: kk, in system32, theres shell32, shell, shell manager, shell style and shellvRTF. Shell manager and shell32 are the only ones that VB6 will add a reference to, and if I add them and select them, I still get the same error,
any other ideas?
PS. ty to all people trying to help, it's greatly apreciated
Aug 21 '07 #7

kadghar
Expert 100+
P: 1,295
ive heard that too, and i also tried shellexecute cause of it, but it still wasnt working, and besides, it doesnt make any sence that shell cant handle spaces cause it was working before ...



ok, i tried that, but i didnt find anything saying missing library, and theres only 4 things checked, so as of now, im gonna try searching on google for the dll file that shell uses, if i cant find anything that way, then i'll just select all the references and test if it works, if it dosnt, then im stuck again, if it does work, then i have to find out which one it is lmao...

EDIT: i found shell's dll, and wadda u no, its called shell32.dll, ima try to get it to work now...

PS. ty to all ppl trying to help, its greatly apreciated
Hi, I dont know how much this helps, but if in the worst case you get stuck, use an object with an internet explorer application in it.
check this out:

VB Forum - On Click

It might be an alternative.
Aug 22 '07 #8

P: 17
Hi, I dont know how much this helps, but if in the worst case you get stuck, use an object with an internet explorer application in it.
check this out:

VB Forum - On Click

It might be an alternative.
well, it does work for internet explorer, but i dont know how to make it work for other programs aswell...

ty

i can use this in the meantime, BUT, i would still like to figure out why i cant use shell anymore
Aug 22 '07 #9

P: 17
That would produce a compile error, not runtime.

I thought somebody was saying recently that Shell couldn't handle a path with spaces in it. Perhaps ShellExecute is better. Have a look at this thread.
i didnt use shell execute from that link, but i did finally find annother source, which had what i needed on it, and i can change it for more than one program

but...

i still wanna no why my shell doesnt work anymore .... and its not cause of spaces in the title, cause i get the same error even with files that dont have spaces
Aug 22 '07 #10

Expert 5K+
P: 8,435
... i still wanna no why my shell doesnt work anymore .... and its not cause of spaces in the title, cause i get the same error even with files that dont have spaces
I'd be really curious to know what went wrong, too. I finally got around to trying your original example, and of course it works. So for some reason your system simply became unable to fire off another process. Could this have been some security measure you weren't aware had been implemented?

Another thought which occurs - if, say, you had something which was being displayed "system modally", then maybe trying to start up anything with the focus causes that error. Could you go back and try the original version again to ensure the problem is still there, then try it without giving it the focus?

That's about it - I'm all guessed out.
Aug 22 '07 #11

P: 17
I'd be really curious to know what went wrong, too. I finally got around to trying your original example, and of course it works. So for some reason your system simply became unable to fire off another process. Could this have been some security measure you weren't aware had been implemented?

Another thought which occurs - if, say, you had something which was being displayed "system modally", then maybe trying to start up anything with the focus causes that error. Could you go back and try the original version again to ensure the problem is still there, then try it without giving it the focus?

That's about it - I'm all guessed out.
For the second part, I thought about that too at one point where I was trying to do everything I could think of to get it to work, I made it as basic as it could possibly be.

For the first part though, that could be what it is, that somehow my security isn’t allowing shell to connect any more, but I haven’t changed anything security-wise recently.

There were 2 things that I did in between when it was and wasn’t working (not including things I do all the time, like msn messenger etc).

One of which was downloading a transparent picturebox user control for use with another program in VB6 (which works amazingly well) and at the time I wasn’t quite sure how to get the user control to work as I had never used it before, so I attempted to put it into system32, and later removed it of course. The second thing I did was download a program made in C++ coding.

Could the user control for VB6 cause an error, even without the trans picbox user control in the program? I know the C++ program is safe, but it does connect to the internet to receive updates, could something inside the program have stoped shell from working by some sort of conflict?

could either of these (or maybe still something else) be causing an error?

Thanks again for trying to help.
Aug 25 '07 #12

P: 4
Hi Guys,

i have a similar kind of problem which i faced while developing Excel VBA Macro.

i have two lines of code in different modules in the same project

1. RetVal = Shell(FILE_PATH & "EXTRACT.BAT")

2. RetVal = Shell(FILE_PATH & "EXTRACT.BAT")


Then 1st shell command works fine. and i did not get any error.

but when i tried to execute the 2nd module which contains the same shell method as in (2), i am getting the error "Invalid Procedure call or argument", Runtime Error 5.

and it also works well when i say "var = Shell("calc.exe")

i am very much confused with the behaviour.

Please anyone in this world can help this.

Thanks

Crapy Coder
Nov 5 '08 #13

P: 2
I started having the same problem with Shell. To begin with all shell's in my program worked, then a few stopped working, there were no changes to my system. Then more and more of them quit working.

My development system is Vista with UAC on, but it has been for over a year before they shell started to crap out. I also did not download any new service packs/ updates to Vista before Shell quit working.

Right now there are still some Shells that work IE

Retval=Shell("notepad.exe",Vbnormalfocus)... works right now
Retval = shell(Mypath$ & "my program.exe") ... worked until yesterday
Retval = shell(Mypath$ & "my other program.exe") ... worked until a month ago
Retval = shell(Mypath$ & "my other other program.exe") ... worked until a 2 months ago

I have searched the net trying to find out why but no joy. I believe it has something to do with Vista UAC. Anyway my solution is to use API with:

ShellExecute Me.hWnd, vbNullString, Mypath$ + "My Prog.exe", vbNullString, vbNullString, 1

It seems to be working now... who knows if it will tomorrow??
Dec 5 '08 #14

P: 1
Hello
I've just registered to say your site is very useful and nicely done!
Thank you very much for your work.

Cheers!

Sorry for offtopic.
Dec 8 '08 #15

P: 3
I was facing the same problem, the shell command was working fine until one day it wasn't. I got Runtime Error '5'.

After testing i determined, that the error wasn't caused by the constant for the WindowStyle (vBMinimized ... or their Integer representation), but by the path variable.

So this would work fine:
Expand|Select|Wrap|Line Numbers
  1. shell calc, vbNormalFocus
but
Expand|Select|Wrap|Line Numbers
  1. shell "c:\my projects\foo.bat", vbNormalFocus"
would not.
The reason for this is the space in "my projects". You have to quote the string by writing
Expand|Select|Wrap|Line Numbers
  1. shell """c:\my projects\foo.bat""", vbNormalFocus"
now it works.

Why it was working the first times, when i didn't quote the string, i dont know. But it is now.
Mar 25 '09 #16

P: 2
This worked for years until yesterday (no spaces in the path):

Shell ("I:\Users\Ellen\CopyPrFiles.bat")

Same Run-time error '5' everybody else is getting.

XP SP3, Excel 2007.

Ideas?
May 5 '09 #17

P: 3
Try

Shell ("""I:\Users\Ellen\CopyPrFiles.bat""")
May 5 '09 #18

P: 2
Adding another set of quotes

Shell ("""I:\Users\Ellen\CopyPrFiles.bat""")

gets me Compile error: Expected: list separator or )
May 5 '09 #19

P: 3
Hi at all!

I've got the same VBA Shell() problem, too.
Calling Shell() from VBA MS Office 2007 on WinXP SP3 and from VBA MS Office 2003 on another WinXP SP3 Computer doesn't work and seems to be broken. (VBA run-time error 5)
The same Office document produces the same error on other WinXP SP3 machines, but works correctly on SP2 computers (tested with MS Office 2007 on WinXP SP2). This is strange and i wonder what's causing this behavior.
I've tried a lot of workarounds to start new processes from VBA without using Shell(), but they all returned an error, like Shell().

Here's my VBA-Code and some explanations:

Prerequisites: C:\test.bat, world readable,writeable,executeable
(Simple batch file with one echo and pause command, works correctly for itself)

The Shell() Method:
Expand|Select|Wrap|Line Numbers
  1. Sub Makro1()
  2. Dim RetVal As Long
  3.     'works well, the IE comes up with google.de
  4.     Shell "C:\Programme\Internet Explorer\Iexplore.exe http://www.google.de", vbMaximizedFocus
  5.     'Invalid procedure call or argument, runtime error 5:
  6.     RetVal = Shell("c:\test.bat")
  7.     'Invalid procedure call or argument, runtime error 5:
  8.     RetVal = Shell(Chr(34) & "c:\test.bat" & Chr(34))
  9.     'works well: 
  10.     RetVal = Shell("c:\windows\system32\notepad.exe", vbNormalNoFocus)
  11.     'works well: 
  12.     RetVal = Shell("c:\windows\system32\notepad.exe", vbMinimizedFocus)
  13.     'works well: 
  14.     RetVal = Shell("C:\WINDOWS\ServicePackFiles\i386\cmd.exe")
  15.     'Invalid procedure call or argument, runtime error 5:
  16.     RetVal = Shell("C:\WINDOWS\System32\cmd.exe")
  17.     'Invalid procedure call or argument, runtime error 5:
  18.     RetVal = Shell("cmd")
  19.     'works well: 
  20.     RetVal = Shell("RUNDLL32.EXE URL.DLL,FileProtocolHandler " & "http://www.google.com", vbMaximizedFocus)
  21.     'works well: 
  22.     RetVal = Shell("RUNDLL32.EXE URL.DLL,FileProtocolHandler " & "c:\test.bat", vbMaximizedFocus)
  23. End Sub
  24.  
The runtime error came up even with d:\test.bat.
I ran Excel and this makro as a domainuser with local administrator rights and as the local administrator itself: the Shell()-error stayed.

Now some workarounds, that work for some files, but not for all, exeactly like the Shell()-call:

WinExec() test:
Expand|Select|Wrap|Line Numbers
  1. Private Declare Function WinExec Lib "kernel32" (ByVal lpCmdLine As String, ByVal nCmdShow As Long) As Long
  2.  
  3. Sub Do_WinExec()
  4.     'works well: 
  5.     WinExec "c:\windows\system32\notepad.exe", 1
  6.     'nothing happens, makro continues: 
  7.     WinExec "c:\test.bat", 1
  8.     'works well: 
  9.     WinExec "C:\WINDOWS\ServicePackFiles\i386\cmd.exe", 1
  10.     'nothing happens, makro continues: 
  11.     WinExec "C:\WINDOWS\System32\cmd.exe", 1
  12. End Sub
  13.  
WinExec() doesn't work, either.



ShellExecute() test:
Expand|Select|Wrap|Line Numbers
  1. Private Declare Function GetActiveWindow Lib "user32.dll" () As Long
  2. Private Declare Function ShellExecute Lib "shell32.dll" _
  3.   Alias "ShellExecuteA" ( _
  4.     ByVal hwnd As Long, _
  5.     ByVal lpOperation As String, _
  6.     ByVal lpFile As String, _
  7.     ByVal lpParameters As String, _
  8.     ByVal lpDirectory As String, _
  9.     ByVal nShowCmd As Long) _
  10. As Long
  11. Private Enum ShellExecute_ShowTypeEnum ' = nShowCmd
  12.     SW_HIDE = 0
  13.     SW_SHOWNORMAL = 1
  14.     SW_SHOWMINIMIZED = 2
  15.     SW_SHOWMAXIMIZED = 3
  16.     SW_SHOWNOACTIVATE = 4
  17.     SW_SHOW = 5
  18.     SW_MINIMIZE = 6
  19.     SW_SHOWMINNOACTIVE = 7
  20.     SW_SHOWNA = 8
  21.     SW_RESTORE = 9
  22.     SW_SHOWDEFAULT = 10
  23.     SW_FORCEMINIMIZE = 11
  24. End Enum
  25. Private Enum ShellExecute_ErrTypeEnum ' = ShellExecute Returnvalue
  26.     ERROR_BAD_FORMAT = 11&  ' Datei ist keine Win32 Anwendung
  27.     SE_ERR_ACCESSDENIED = 5 ' Zugriff verweigert
  28.     SE_ERR_ASSOCINCOMPLETE = 27 ' Datei-Assoziation ist unvollständig
  29.     SE_ERR_DDEBUSY = 30 ' DDE ist nicht bereit
  30.     SE_ERR_DDEFAIL = 29 ' DDE-Vorgang gescheitert
  31.     SE_ERR_DDETIMEOUT = 28  ' DDE-Zeitlimit wurde erreicht
  32.     SE_ERR_DLLNOTFOUND = 32 ' benötigte DLL wurde nicht gefunden
  33.     SE_ERR_FNF = 2  ' Datei wurde nicht gefunden
  34.     SE_ERR_NOASSOC = 31 ' Datei ist nicht Assoziiert
  35.     SE_ERR_OOM = 8  ' Nicht genügend Speicher
  36.     SE_ERR_PNF = 3  ' Pfad wurde nicht gefunden
  37.     SE_ERR_SHARE = 2    ' Sharing-Verletzung
  38. End Enum
  39.  
  40. Sub Do_ShellExecute()
  41. Dim sDir
  42. Dim sFileName
  43. Dim sParameters
  44. Dim RetVal
  45. Dim hwnd
  46.        'with these parameters, ShellExecute() will succeed and notepad will show c:\test.bat
  47.     'sDir = "c:\windows\system32\"
  48.     'sFileName = "c:\windows\system32\notepad.exe"
  49.     'sParameters = "c:\test.bat"
  50.  
  51.     'but with these parameters, ShellExecute() will return error 5, SE_ERR_ACCESSDENIED
  52.     sDir = "c:\"
  53.     sFileName = "c:\test.bat"
  54.     sParameters = "abc"
  55.     hwnd = GetActiveWindow()
  56.     RetVal = ShellExecute(hwnd, "open", sFileName, sParameters, sDir, ShellExecute_ShowTypeEnum.SW_SHOWNORMAL)
  57.     If RetVal = ShellExecute_ErrTypeEnum.SE_ERR_NOASSOC Then
  58.         MsgBox "No file association found - Keine Programmzuweisung gefunden."
  59.     ElseIf RetVal < 33 Then
  60.         MsgBox "An error occurred, error number: " & RetVal
  61.     End If
  62. End Sub
  63.  
ShellExecute() doesn't work, either.



WScript.Shell test:
Expand|Select|Wrap|Line Numbers
  1. Private Sub WSH_Shell()
  2. Dim WSH_Shell
  3. Set WSH_Shell = CreateObject("WScript.Shell")
  4.  
  5.     'works well:
  6.     WSH_Shell.Run ("c:\windows\system32\notepad.exe"), , True  ' execute command and wait for it to finish
  7.     'works well:
  8.     WSH_Shell.Run ("C:\WINDOWS\ServicePackFiles\i386\cmd.exe"), , True  ' execute command and wait for it to finish
  9.     ' returns runtime error 70, access denied
  10.     WSH_Shell.Run ("C:\WINDOWS\System32\cmd.exe"), , True  ' execute command and wait for it to finish
  11.     ' returns runtime error 70, access denied
  12.     WSH_Shell.Run ("c:\test.bat"), , True  ' execute command and wait for it to finish
  13. End Sub
  14.  
Shell.Run() doesn't work, either.


The CreateProcess() example from MS (http://support.microsoft.com/kb/129797) works fine with calc.exe, but not with all the problem executeables and batch files named in Makro1 above!
The returnvalue of CreateProcess() is 0, when it fails. The VBA Err-object just reports an error number 5 (Err.LastDllError = 5, everything else of Err is empty)



Conclusion:
Shell(), WinExec, ShellExecute(), WScript.Shell and CreateProcess() were not able to start certain, but always the same executeables/batch files.
This applies to VBA code (tested with Excel and Word 2007) on WinXP SP3 32Bit machines. This code works fine on WinXP 32Bit SP2 machines!! :--o ??

I was trying to find a missconfiguration with dcomcnfg.exe, but i must confess, i don't know what all this stuff means exactly... :--/

Perhaps someone who had/has the same Shell() error could help us?
Thank you in advance!
Jul 30 '09 #20

P: 3
Good news!

After a few days of research in this this VBA Shell() run-time error 5, we figured our virus scanner out to be the culprit. We deactivated the Trend Micro Virus Scanner at our workstations and all my tested methods for executing a file from VBA worked well again.

I wonder why it took us so long to blame the virus scanner and test it...
I hope at least this hint helps others.

Bye!

EDIT:
P.S.: the Virus Scanner did never complain about any VBA-actions, there were no notifications
Aug 4 '09 #21

kadghar
Expert 100+
P: 1,295
Thanks HannesBeh,

I´m pretty sure many will find useful your experience with Shell and the Virus Scanner.
Aug 4 '09 #22

P: 3
I hope my postings are usefull for one or another.
For completeness:
Our Trend Micro Virus Scanner has been auto updated (in August 2009, Version 08/07/2009, 08/05/2009) and now those Shell()-errors are gone, even with activated virus scanner.
Aug 10 '09 #23

P: n/a
Thx guys, this has been driving me mad, shell suddenly stopped working with runtime error 5.
Change to triple quotes resolved the problem.
Thankyou

Dim retval
retval = Shell("""C:\Users\Helter\Zebra\Print.bat""", vbHide)
Oct 8 '10 #24

P: 1
@HelterSkelter
OLD POST BUT A GOOD ONE .. THIS POST HELPED ME.
I note i never have the problem on windows 7 or 10.
but i tried to run the code on my old XP laptop
Oct 23 '15 #25

Post your reply

Sign in to post your reply or Sign up for a free account.