Shell run-time error 5, invalid procedure call or argument

[dennijr]

ok, shell always used to be easy for me, now its starting to get annoying cause i dont know wats wrong

heres the simplist code possible:

Private Sub IExplorer_Click()
a = Shell("C:\Program Files\Internet Explorer\Iexplore.exe http://www.google.ca", vbMaximizedFocus)
End Sub

i have a program thats been on my desktop for about a month now that uses these 3 lines of code(supposed to be 3 lines but it doesnt quite fit) and its been working perfectly up till today. I attempted to run it, then when it got to this part, it highlighted

a = Shell("C:\Program Files\Internet Explorer\Iexplore.exe http://www.google.ca", vbMaximizedFocus)

and a messagebox appeared saying run-time error 5, invalid procedure call or argument. I tried making an even simpler program that just opened notepad during form_load, but it still didnt work, same error. It doesnt make any sence, all that stuff used to work perfectly.

[kadghar]

ok, shell always used to be easy for me, now its starting to get annoying cause i dont know wats wrong

heres the simplist code possible:

Private Sub IExplorer_Click()
a = Shell("C:\Program Files\Internet Explorer\Iexplore.exe http://www.google.ca", vbMaximizedFocus)
End Sub

i have a program thats been on my desktop for about a month now that uses these 3 lines of code(supposed to be 3 lines but it doesnt quite fit) and its been working perfectly up till today. I attempted to run it, then when it got to this part, it highlighted

a = Shell("C:\Program Files\Internet Explorer\Iexplore.exe http://www.google.ca", vbMaximizedFocus)

and a messagebox appeared saying run-time error 5, invalid procedure call or argument. I tried making an even simpler program that just opened notepad during form_load, but it still didnt work, same error. It doesnt make any sence, all that stuff used to work perfectly.

Hi,
I think this is because of the second argument, "vbmaximizedfocus" maybe you don have the constants set, why dont you try using the constan's value instead, something like

a = Shell("C:\Program Files\Internet Explorer\Iexplore.exe http://www.google.ca", 3)

hope that helps

[dennijr]

Hi,
I think this is because of the second argument, "vbmaximizedfocus" maybe you don have the constants set, why dont you try using the constan's value instead, something like

a = Shell("C:\Program Files\Internet Explorer\Iexplore.exe http://www.google.ca", 3)

hope that helps

nope, didnt work = ( , thanks for making an attempt tho

[Killer42]

I think this is because of the second argument, "vbmaximizedfocus" maybe you don have the constants set...

That would produce a compile error, not runtime.

I thought somebody was saying recently that Shell couldn't handle a path with spaces in it. Perhaps ShellExecute is better. Have a look at this thread.

[pureenhanoi]

ok, shell always used to be easy for me, now its starting to get annoying cause i dont know wats wrong

heres the simplist code possible:

Private Sub IExplorer_Click()
a = Shell("C:\Program Files\Internet Explorer\Iexplore.exe http://www.google.ca", vbMaximizedFocus)
End Sub

i have a program thats been on my desktop for about a month now that uses these 3 lines of code(supposed to be 3 lines but it doesnt quite fit) and its been working perfectly up till today. I attempted to run it, then when it got to this part, it highlighted

a = Shell("C:\Program Files\Internet Explorer\Iexplore.exe http://www.google.ca", vbMaximizedFocus)

and a messagebox appeared saying run-time error 5, invalid procedure call or argument. I tried making an even simpler program that just opened notepad during form_load, but it still didnt work, same error. It doesnt make any sence, all that stuff used to work perfectly.

I sometime meet this type of error. (Worked-code goes fault). I tried resolving problem and found the reason: missing one or more library in preference.
On the Project menu, choose Preference. One dialoge appears show all preferences of your project. If you see atleast one line have "...Missing...library...", so it is the fault.
If this library doesnot need for project, so simply un-check the library and click OK. If this really need, you can see the path to the ".dll" file. If the file is missing, u need copy one file to that location, or maybe install some other soft-wares that provide this "dll" file.

[Killer42]

I sometime meet this type of error. (Worked-code goes fault). I tried resolving problem and found the reason: missing one or more library in preference...

Are you sure you don't mean reference?

[dennijr]

That would produce a compile error, not runtime.

I thought somebody was saying recently that Shell couldn't handle a path with spaces in it. Perhaps ShellExecute is better. Have a look at this thread.

I've heard that too, and I also tried ShellExecute cause of it, but it still wasn't working, and besides, it doesn't make any sense that Shell can't handle spaces cause it was working before ...

I sometime meet this type of error. (Worked-code goes fault). I tried resolving problem and found the reason: missing one or more library in preference.
On the Project menu, choose Preference. One dialoge appears show all preferences of your project. If you see atleast one line have "...Missing...library...", so it is the fault.
If this library doesnot need for project, so simply un-check the library and click OK. If this really need, you can see the path to the ".dll" file. If the file is missing, u need copy one file to that location, or maybe install some other soft-wares that provide this "dll" file.

Ok, I tried that, but I didn't find anything saying missing library, and there's only 4 things checked, so as of now, I'm gonna try searching on Google for the dll file that shell uses, if I can't find anything that way, then I'll just select all the references and test if it works, if it doesn't, then I'm stuck again, if it does work, then I have to find out which one it is lmao...

EDIT: kk, in system32, theres shell32, shell, shell manager, shell style and shellvRTF. Shell manager and shell32 are the only ones that VB6 will add a reference to, and if I add them and select them, I still get the same error,
any other ideas?
PS. ty to all people trying to help, it's greatly apreciated

[kadghar]

ive heard that too, and i also tried shellexecute cause of it, but it still wasnt working, and besides, it doesnt make any sence that shell cant handle spaces cause it was working before ...



ok, i tried that, but i didnt find anything saying missing library, and theres only 4 things checked, so as of now, im gonna try searching on google for the dll file that shell uses, if i cant find anything that way, then i'll just select all the references and test if it works, if it dosnt, then im stuck again, if it does work, then i have to find out which one it is lmao...

EDIT: i found shell's dll, and wadda u no, its called shell32.dll, ima try to get it to work now...

PS. ty to all ppl trying to help, its greatly apreciated

Hi, I dont know how much this helps, but if in the worst case you get stuck, use an object with an internet explorer application in it.
check this out:

VB Forum - On Click

It might be an alternative.

[dennijr]

Hi, I dont know how much this helps, but if in the worst case you get stuck, use an object with an internet explorer application in it.
check this out:

VB Forum - On Click

It might be an alternative.

well, it does work for internet explorer, but i dont know how to make it work for other programs aswell...

ty

i can use this in the meantime, BUT, i would still like to figure out why i cant use shell anymore

[dennijr]

That would produce a compile error, not runtime.

I thought somebody was saying recently that Shell couldn't handle a path with spaces in it. Perhaps ShellExecute is better. Have a look at this thread.

i didnt use shell execute from that link, but i did finally find annother source, which had what i needed on it, and i can change it for more than one program

but...

i still wanna no why my shell doesnt work anymore .... and its not cause of spaces in the title, cause i get the same error even with files that dont have spaces

[Killer42]

... i still wanna no why my shell doesnt work anymore .... and its not cause of spaces in the title, cause i get the same error even with files that dont have spaces

I'd be really curious to know what went wrong, too. I finally got around to trying your original example, and of course it works. So for some reason your system simply became unable to fire off another process. Could this have been some security measure you weren't aware had been implemented?

Another thought which occurs - if, say, you had something which was being displayed "system modally", then maybe trying to start up anything with the focus causes that error. Could you go back and try the original version again to ensure the problem is still there, then try it without giving it the focus?

That's about it - I'm all guessed out.

[dennijr]

I'd be really curious to know what went wrong, too. I finally got around to trying your original example, and of course it works. So for some reason your system simply became unable to fire off another process. Could this have been some security measure you weren't aware had been implemented?

Another thought which occurs - if, say, you had something which was being displayed "system modally", then maybe trying to start up anything with the focus causes that error. Could you go back and try the original version again to ensure the problem is still there, then try it without giving it the focus?

That's about it - I'm all guessed out.

For the second part, I thought about that too at one point where I was trying to do everything I could think of to get it to work, I made it as basic as it could possibly be.

For the first part though, that could be what it is, that somehow my security isn’t allowing shell to connect any more, but I haven’t changed anything security-wise recently.

There were 2 things that I did in between when it was and wasn’t working (not including things I do all the time, like msn messenger etc).

One of which was downloading a transparent picturebox user control for use with another program in VB6 (which works amazingly well) and at the time I wasn’t quite sure how to get the user control to work as I had never used it before, so I attempted to put it into system32, and later removed it of course. The second thing I did was download a program made in C++ coding.

Could the user control for VB6 cause an error, even without the trans picbox user control in the program? I know the C++ program is safe, but it does connect to the internet to receive updates, could something inside the program have stoped shell from working by some sort of conflict?

could either of these (or maybe still something else) be causing an error?

Thanks again for trying to help.

[crapycoder]

Hi Guys,

i have a similar kind of problem which i faced while developing Excel VBA Macro.

i have two lines of code in different modules in the same project

1. RetVal = Shell(FILE_PATH & "EXTRACT.BAT")

2. RetVal = Shell(FILE_PATH & "EXTRACT.BAT")


Then 1st shell command works fine. and i did not get any error.

but when i tried to execute the 2nd module which contains the same shell method as in (2), i am getting the error "Invalid Procedure call or argument", Runtime Error 5.

and it also works well when i say "var = Shell("calc.exe")

i am very much confused with the behaviour.

Please anyone in this world can help this.

Thanks

Crapy Coder

[wakerunner]

I started having the same problem with Shell. To begin with all shell's in my program worked, then a few stopped working, there were no changes to my system. Then more and more of them quit working.

My development system is Vista with UAC on, but it has been for over a year before they shell started to crap out. I also did not download any new service packs/ updates to Vista before Shell quit working.

Right now there are still some Shells that work IE

Retval=Shell("notepad.exe",Vbnormalfocus)... works right now
Retval = shell(Mypath$ & "my program.exe") ... worked until yesterday
Retval = shell(Mypath$ & "my other program.exe") ... worked until a month ago
Retval = shell(Mypath$ & "my other other program.exe") ... worked until a 2 months ago

I have searched the net trying to find out why but no joy. I believe it has something to do with Vista UAC. Anyway my solution is to use API with:

ShellExecute Me.hWnd, vbNullString, Mypath$ + "My Prog.exe", vbNullString, vbNullString, 1

It seems to be working now... who knows if it will tomorrow??

[DrardyLarmbab]

Hello
I've just registered to say your site is very useful and nicely done!
Thank you very much for your work.

Cheers!

Sorry for offtopic.

[elToro]

I was facing the same problem, the shell command was working fine until one day it wasn't. I got Runtime Error '5'.

After testing i determined, that the error wasn't caused by the constant for the WindowStyle (vBMinimized ... or their Integer representation), but by the path variable.

So this would work fine:

shell calc, vbNormalFocus

but

shell "c:\my projects\foo.bat", vbNormalFocus"

would not.
The reason for this is the space in "my projects". You have to quote the string by writing

shell """c:\my projects\foo.bat""", vbNormalFocus"

now it works.

Why it was working the first times, when i didn't quote the string, i dont know. But it is now.

[retsyboss]

This worked for years until yesterday (no spaces in the path):

Shell ("I:\Users\Ellen\CopyPrFiles.bat")

Same Run-time error '5' everybody else is getting.

XP SP3, Excel 2007.

Ideas?

[elToro]

Try

Shell ("""I:\Users\Ellen\CopyPrFiles.bat""")

[retsyboss]

Adding another set of quotes

Shell ("""I:\Users\Ellen\CopyPrFiles.bat""")

gets me Compile error: Expected: list separator or )

[HannesBeh]

Hi at all!

I've got the same VBA Shell() problem, too.
Calling Shell() from VBA MS Office 2007 on WinXP SP3 and from VBA MS Office 2003 on another WinXP SP3 Computer doesn't work and seems to be broken. (VBA run-time error 5)
The same Office document produces the same error on other WinXP SP3 machines, but works correctly on SP2 computers (tested with MS Office 2007 on WinXP SP2). This is strange and i wonder what's causing this behavior.
I've tried a lot of workarounds to start new processes from VBA without using Shell(), but they all returned an error, like Shell().

Here's my VBA-Code and some explanations:

Prerequisites: C:\test.bat, world readable,writeable,executeable
(Simple batch file with one echo and pause command, works correctly for itself)

The Shell() Method:

Sub Makro1()
Dim RetVal As Long
    'works well, the IE comes up with google.de
    Shell "C:\Programme\Internet Explorer\Iexplore.exe http://www.google.de", vbMaximizedFocus
    'Invalid procedure call or argument, runtime error 5:
    RetVal = Shell("c:\test.bat")
    'Invalid procedure call or argument, runtime error 5:
    RetVal = Shell(Chr(34) & "c:\test.bat" & Chr(34))
    'works well: 
    RetVal = Shell("c:\windows\system32\notepad.exe", vbNormalNoFocus)
    'works well: 
    RetVal = Shell("c:\windows\system32\notepad.exe", vbMinimizedFocus)
    'works well: 
    RetVal = Shell("C:\WINDOWS\ServicePackFiles\i386\cmd.exe")
    'Invalid procedure call or argument, runtime error 5:
    RetVal = Shell("C:\WINDOWS\System32\cmd.exe")
    'Invalid procedure call or argument, runtime error 5:
    RetVal = Shell("cmd")
    'works well: 
    RetVal = Shell("RUNDLL32.EXE URL.DLL,FileProtocolHandler " & "http://www.google.com", vbMaximizedFocus)
    'works well: 
    RetVal = Shell("RUNDLL32.EXE URL.DLL,FileProtocolHandler " & "c:\test.bat", vbMaximizedFocus)
End Sub
 

The runtime error came up even with d:\test.bat.
I ran Excel and this makro as a domainuser with local administrator rights and as the local administrator itself: the Shell()-error stayed.

Now some workarounds, that work for some files, but not for all, exeactly like the Shell()-call:

WinExec() test:

Private Declare Function WinExec Lib "kernel32" (ByVal lpCmdLine As String, ByVal nCmdShow As Long) As Long
 
Sub Do_WinExec()
    'works well: 
    WinExec "c:\windows\system32\notepad.exe", 1
    'nothing happens, makro continues: 
    WinExec "c:\test.bat", 1
    'works well: 
    WinExec "C:\WINDOWS\ServicePackFiles\i386\cmd.exe", 1
    'nothing happens, makro continues: 
    WinExec "C:\WINDOWS\System32\cmd.exe", 1
End Sub
 

WinExec() doesn't work, either.



ShellExecute() test:

Private Declare Function GetActiveWindow Lib "user32.dll" () As Long
Private Declare Function ShellExecute Lib "shell32.dll" _
  Alias "ShellExecuteA" ( _
    ByVal hwnd As Long, _
    ByVal lpOperation As String, _
    ByVal lpFile As String, _
    ByVal lpParameters As String, _
    ByVal lpDirectory As String, _
    ByVal nShowCmd As Long) _
As Long
Private Enum ShellExecute_ShowTypeEnum ' = nShowCmd
    SW_HIDE = 0
    SW_SHOWNORMAL = 1
    SW_SHOWMINIMIZED = 2
    SW_SHOWMAXIMIZED = 3
    SW_SHOWNOACTIVATE = 4
    SW_SHOW = 5
    SW_MINIMIZE = 6
    SW_SHOWMINNOACTIVE = 7
    SW_SHOWNA = 8
    SW_RESTORE = 9
    SW_SHOWDEFAULT = 10
    SW_FORCEMINIMIZE = 11
End Enum
Private Enum ShellExecute_ErrTypeEnum ' = ShellExecute Returnvalue
    ERROR_BAD_FORMAT = 11&  ' Datei ist keine Win32 Anwendung
    SE_ERR_ACCESSDENIED = 5 ' Zugriff verweigert
    SE_ERR_ASSOCINCOMPLETE = 27 ' Datei-Assoziation ist unvollständig
    SE_ERR_DDEBUSY = 30 ' DDE ist nicht bereit
    SE_ERR_DDEFAIL = 29 ' DDE-Vorgang gescheitert
    SE_ERR_DDETIMEOUT = 28  ' DDE-Zeitlimit wurde erreicht
    SE_ERR_DLLNOTFOUND = 32 ' benötigte DLL wurde nicht gefunden
    SE_ERR_FNF = 2  ' Datei wurde nicht gefunden
    SE_ERR_NOASSOC = 31 ' Datei ist nicht Assoziiert
    SE_ERR_OOM = 8  ' Nicht genügend Speicher
    SE_ERR_PNF = 3  ' Pfad wurde nicht gefunden
    SE_ERR_SHARE = 2    ' Sharing-Verletzung
End Enum
 
Sub Do_ShellExecute()
Dim sDir
Dim sFileName
Dim sParameters
Dim RetVal
Dim hwnd
       'with these parameters, ShellExecute() will succeed and notepad will show c:\test.bat
    'sDir = "c:\windows\system32\"
    'sFileName = "c:\windows\system32\notepad.exe"
    'sParameters = "c:\test.bat"
 
    'but with these parameters, ShellExecute() will return error 5, SE_ERR_ACCESSDENIED
    sDir = "c:\"
    sFileName = "c:\test.bat"
    sParameters = "abc"
    hwnd = GetActiveWindow()
    RetVal = ShellExecute(hwnd, "open", sFileName, sParameters, sDir, ShellExecute_ShowTypeEnum.SW_SHOWNORMAL)
    If RetVal = ShellExecute_ErrTypeEnum.SE_ERR_NOASSOC Then
        MsgBox "No file association found - Keine Programmzuweisung gefunden."
    ElseIf RetVal < 33 Then
        MsgBox "An error occurred, error number: " & RetVal
    End If
End Sub
 

ShellExecute() doesn't work, either.



WScript.Shell test:

Private Sub WSH_Shell()
Dim WSH_Shell
Set WSH_Shell = CreateObject("WScript.Shell")
 
    'works well:
    WSH_Shell.Run ("c:\windows\system32\notepad.exe"), , True  ' execute command and wait for it to finish
    'works well:
    WSH_Shell.Run ("C:\WINDOWS\ServicePackFiles\i386\cmd.exe"), , True  ' execute command and wait for it to finish
    ' returns runtime error 70, access denied
    WSH_Shell.Run ("C:\WINDOWS\System32\cmd.exe"), , True  ' execute command and wait for it to finish
    ' returns runtime error 70, access denied
    WSH_Shell.Run ("c:\test.bat"), , True  ' execute command and wait for it to finish
End Sub
 

Shell.Run() doesn't work, either.


The CreateProcess() example from MS (http://support.microsoft.com/kb/129797) works fine with calc.exe, but not with all the problem executeables and batch files named in Makro1 above!
The returnvalue of CreateProcess() is 0, when it fails. The VBA Err-object just reports an error number 5 (Err.LastDllError = 5, everything else of Err is empty)



Conclusion:
Shell(), WinExec, ShellExecute(), WScript.Shell and CreateProcess() were not able to start certain, but always the same executeables/batch files.
This applies to VBA code (tested with Excel and Word 2007) on WinXP SP3 32Bit machines. This code works fine on WinXP 32Bit SP2 machines!! :--o ??

I was trying to find a missconfiguration with dcomcnfg.exe, but i must confess, i don't know what all this stuff means exactly... :--/

Perhaps someone who had/has the same Shell() error could help us?
Thank you in advance!

[HannesBeh]

Good news!

After a few days of research in this this VBA Shell() run-time error 5, we figured our virus scanner out to be the culprit. We deactivated the Trend Micro Virus Scanner at our workstations and all my tested methods for executing a file from VBA worked well again.

I wonder why it took us so long to blame the virus scanner and test it...
I hope at least this hint helps others.

Bye!

EDIT:
P.S.: the Virus Scanner did never complain about any VBA-actions, there were no notifications

[kadghar]

Thanks HannesBeh,

I´m pretty sure many will find useful your experience with Shell and the Virus Scanner.

[HannesBeh]

I hope my postings are usefull for one or another.
For completeness:
Our Trend Micro Virus Scanner has been auto updated (in August 2009, Version 08/07/2009, 08/05/2009) and now those Shell()-errors are gone, even with activated virus scanner.

[HelterSkelter]

Thx guys, this has been driving me mad, shell suddenly stopped working with runtime error 5.
Change to triple quotes resolved the problem.
Thankyou

Dim retval
retval = Shell("""C:\Users\Helter\Zebra\Print.bat""", vbHide)

[CKRUM]

@HelterSkelter
OLD POST BUT A GOOD ONE .. THIS POST HELPED ME.
I note i never have the problem on windows 7 or 10.
but i tried to run the code on my old XP laptop

[netgroover]

@elToro
I had assigned path and filename to a variable and had not had an issue with this code for ten years! However, about a week ago this started throwing the Error 5. I had to type the literal path and filename and surround with quotes as you suggested, and it worked. I wonder what MS did that caused this change in behavior, and whether it will be patched/fixed in the future.