CryptDecrypt 'System.ExecutionEngineException'

[ShadowLocke]

Hi!

I'm having a problem calling CryptDecrypt from .Net.

Even thoug I call from inside a try catch block execution stops on the call with error "Exception of type 'System.ExecutionEngineException' was thrown." with no extra information after that..

Has anyone run into a problem like this before?

[DllImport("advapi32.dll", EntryPoint = "CryptAcquireContextW", CharSet = CharSet.Unicode, SetLastError = true)]
private static extern bool CryptAcquireContext(ref IntPtr hProv, string pszContainer, string pszProvider, uint dwProvType, uint dwFlags);
 
[DllImport("advapi32.dll", SetLastError = true)]
private static extern bool CryptReleaseContext(IntPtr hProv, uint dwFlags);
 
[DllImport("advapi32.dll", SetLastError = true)]
private static extern bool CryptDecrypt(IntPtr hKey, uint hHash, uint Final, uint dwFlags, [MarshalAs(UnmanagedType.LPWStr)]ref String pbData, ref uint pdwDataLen);
 
//Tried these as well..
//[DllImport("advapi32.dll", SetLastError = true)]
//private static extern bool CryptDecrypt(IntPtr hKey, uint hHash, uint Final, uint dwFlags, ref String pbData, ref uint pdwDataLen);
 
//[DllImport("advapi32.dll", SetLastError = true)]
//private static extern bool CryptDecrypt(IntPtr hKey, uint hHash, uint Final, uint dwFlags, ref byte[] pbData, ref uint pdwDataLen);
 
//[DllImport("advapi32.dll", SetLastError = true)]
//private static extern bool CryptDecrypt(IntPtr hKey, uint hHash, uint Final, uint dwFlags, IntPtr pbData, ref uint pdwDataLen);
 
 
[DllImport("advapi32.dll", SetLastError = true)]
private static extern bool CryptCreateHash(IntPtr hProv, uint Algid, IntPtr hKey, uint dwFlags, ref IntPtr phHash);
 
[DllImport("advapi32.dll", SetLastError = true)]
private static extern bool CryptHashData(IntPtr hHash, String pbData, uint dwDataLen, uint dwFlags);
 
[DllImport("advapi32.dll", SetLastError = true)]
private static extern bool CryptDestroyHash(IntPtr hHash);
 
[DllImport("advapi32.dll", SetLastError = true)]
private static extern bool CryptDeriveKey(IntPtr hProv, uint Algid, IntPtr hBaseData, uint dwFlags, ref IntPtr phKey);
 
[DllImport("advapi32.dll", SetLastError = true)]
private static extern bool CryptDestroyKey(IntPtr hKey);

...
 
if (!CryptAcquireContext(ref hCryptProv, "JustTesting", "Microsoft Enhanced RSA and AES Cryptographic Provider (Prototype)", 24, 0))
   if (!CryptAcquireContext(ref hCryptProv, "JustTesting", "Microsoft Enhanced RSA and AES Cryptographic Provider (Prototype)", 24, 8))
         throw new Exception("Error creating CSP handle");
 
if (!CryptCreateHash(hCryptProv, 32772, IntPtr.Zero, 0, ref hHash))
   throw new Exception("Error creating hash.");
 
if (!CryptHashData(hHash, ls_key, 256, 0))
   throw new Exception("Error creating hash.");
 
if (!CryptDeriveKey(hCryptProv, 26128, hHash, 0, ref hKey))
   throw new Exception("Error creating key.");
 
if (!CryptDestroyHash(hHash))
   throw new Exception("Error destroying hash.");
 
hHash = IntPtr.Zero;
 
string ls = "<encrypted string>"
uint length = (uint)ls.Length * 2; //Unicode
 
if (!CryptDecrypt(hKey, 0, 1, 0, ref ls, ref length))
   throw new Exception("Unable to decrypt " + Marshal.GetLastWin32Error().ToString()); //never get here
 
//or here..
...
 

[tlhintoq]

Why are you doing all this through DLL imports instead of the native .NET cryptography services?

[ShadowLocke]

Its been implemented in an older application that it needs to be compatible with and I dont know how to translate the above into the .NET equalvalent

[PRR]

Try using

catch
{
}
 

This is for catching unmanaged exceptions... if i am right...
Check for right data types .Also look into unsafe for use of pointers...

[ShadowLocke]

just tried using

//catch //Wants a try..
//{
//CryptDecrypt(lp_key, 0, 1, 0, ref ls_temp, ref lul_length)
//}
 
try
{
CryptDecrypt(lp_key, 0, 1, 0, ref ls_temp, ref lul_length)
}
catch { }

It gives the same error and does not catch.

Im pretty sure I've got the right datatypes there..

http://msdn.microsoft.com/en-us/libr...13(VS.85).aspx
BOOL WINAPI CryptDecrypt(
__in HCRYPTKEY hKey,
__in HCRYPTHASH hHash,
__in BOOL Final,
__in DWORD dwFlags,
__inout BYTE *pbData,
__inout DWORD *pdwDataLen
);

IntPtr for the first two, boolean for the second (Not reflected in post but was tried)

Such that..defining CryptDecrypt as:

[DllImport("advapi32.dll", SetLastError = true, CharSet = CharSet.Unicode)]
            private static extern bool CryptDecrypt(IntPtr hKey, IntPtr hHash, bool Final, uint dwFlags, ref Byte[] pbData, ref uint pdwDataLen);

Still gives the same uncatchable error.

[PRR]

My last guess would be using unsafe tags for your code....