Connecting Tech Pros Worldwide Forums | Help | Site Map

Q) Accessing secure websites from a Java application

Generic Usenet Account
Guest
 
Posts: n/a
#1: Jul 17 '05
I am trying to create a Java application that reads a list of URLs
from a file and stores their contents on the local file system. I
have succeeded in accessing normal websites, but I am unable to access
the secured websites (using the HTTPS protocol) using this approach.

I would greatly appreciate if someone could suggest a way out. I have
looked at the HttpsURLConnection class, but unfortunately this class
is abstract.

[My apolgies to readers of comp.lang.java and comp.sources.d since
this posting may appear to be a repost of my previous posting
(although it is not) --- Bhat]

My source code follows:

/////////////////// Source code begin /////////////////////

// This program reads a list of URLs to access and store on the local
// file system from a file. The name of the file is passed as the
// first command line argument. Each URL is on a separate line.
// Lines beginning with the '#' character are treated as blanks and
// are skipped.
//
import java.io.*;
import java.net.*;
import java.security.*;


class WebsiteLoader
{
public static char replaceChar = '~';

public static void main(String argv[]) throws IOException
{
// The following two lines were suggested by the following
website:
// http://www.javaworld.com/javaworld/j...javatip96.html
// They help in suppressing the java.net.MalformedURLException
System.setProperty("java.protocol.handler.pkgs",
"com.sun.net.ssl.internal.www.protocol");
Security.addProvider(new com.sun.net.ssl.internal.ssl.Provider());

BufferedReader br;
String origName;

if(argv.length != 0)
{
br = new BufferedReader(new FileReader(argv[0]));

// Read URLs from the file. Skip blank lines and lines
beginning
// with the '#' character.
for(;;)
{
origName = br.readLine();
if(origName == null)
break;

origName = origName.trim();

if(origName.length() == 0)
continue;

if(origName.charAt(0) == '#')
continue;

URL url = new URL(origName);
if(url == null)
continue;

BufferedReader bufRdr = new BufferedReader(new
InputStreamReader(url.openStream()));

// The name of the file to which the website contents are
written
// is derived from the URL by substituting the following
characters
// with some "non-offending" character:
// \,/,:,*,?,",<,>,|

String modName = origName;
modName = modName.replace('\\', replaceChar);
modName = modName.replace('/', replaceChar);
modName = modName.replace(':', replaceChar);
modName = modName.replace('*', replaceChar);
modName = modName.replace('?', replaceChar);
modName = modName.replace('"', replaceChar);
modName = modName.replace('<', replaceChar);
modName = modName.replace('>', replaceChar);
modName = modName.replace('|', replaceChar);

FileWriter fWriter = new FileWriter(modName);


System.out.println("Writing contents of " + origName + " to "
+
"the following file: " + modName);
for(;;)
{
String thisLine = bufRdr.readLine();
if(thisLine == null)
break;

fWriter.write(thisLine);
}
}
}
}
}

/////////////////// Source code end //////////////////////

Tom N
Guest
 
Posts: n/a
#2: Jul 17 '05

re: Q) Accessing secure websites from a Java application


See http://sunsolve.sun.com/pub-cgi/retr...salert%2F57436

The error is because the Class 3 and Class 2 root certificates installed in
your version of Java have expired. It says you can either download the
latest JRE or JDK or you can download certificates from verisign and install
them.

I tried your code and the following is the status of a list of URLs before I
updated the certificate. I am using JDK 1.4.1_02-b06 by the way. This list
is taken from the Javaworld article you referenced.

https://www.verisign.com/ failed
https://happiness.dhs.org/ failed
https://www.microsoft.com worked
https://www.sun.com failed (IIRC)
https://www.ftc.gov failed

Interestingly, those that failed did not all have the same exception thrown.

I updated the cert's as detailed in the sunsolve article and now all of the
URLs worked except dhs.org, which is because it uses a self-signed
certifcate (not using a Verisign or similar root certificate).

"Generic Usenet Account" wrote:[color=blue]
> I am trying to create a Java application that reads a list of URLs
> from a file and stores their contents on the local file system. I
> have succeeded in accessing normal websites, but I am unable to access
> the secured websites (using the HTTPS protocol) using this approach.
>
> I would greatly appreciate if someone could suggest a way out. I have
> looked at the HttpsURLConnection class, but unfortunately this class
> is abstract.
>
> [My apolgies to readers of comp.lang.java and comp.sources.d since
> this posting may appear to be a repost of my previous posting
> (although it is not) --- Bhat]
>
> My source code follows:
>
> /////////////////// Source code begin /////////////////////
>
> // This program reads a list of URLs to access and store on the local
> // file system from a file. The name of the file is passed as the
> // first command line argument. Each URL is on a separate line.
> // Lines beginning with the '#' character are treated as blanks and
> // are skipped.
> //
> import java.io.*;
> import java.net.*;
> import java.security.*;
>
>
> class WebsiteLoader
> {
> public static char replaceChar = '~';
>
> public static void main(String argv[]) throws IOException
> {
> // The following two lines were suggested by the following
> website:
> // http://www.javaworld.com/javaworld/j...javatip96.html
> // They help in suppressing the java.net.MalformedURLException
> System.setProperty("java.protocol.handler.pkgs",
> "com.sun.net.ssl.internal.www.protocol");
> Security.addProvider(new com.sun.net.ssl.internal.ssl.Provider());
>
> BufferedReader br;
> String origName;
>
> if(argv.length != 0)
> {
> br = new BufferedReader(new FileReader(argv[0]));
>
> // Read URLs from the file. Skip blank lines and lines
> beginning
> // with the '#' character.
> for(;;)
> {
> origName = br.readLine();
> if(origName == null)
> break;
>
> origName = origName.trim();
>
> if(origName.length() == 0)
> continue;
>
> if(origName.charAt(0) == '#')
> continue;
>
> URL url = new URL(origName);
> if(url == null)
> continue;
>
> BufferedReader bufRdr = new BufferedReader(new
> InputStreamReader(url.openStream()));
>
> // The name of the file to which the website contents are
> written
> // is derived from the URL by substituting the following
> characters
> // with some "non-offending" character:
> // \,/,:,*,?,",<,>,|
>
> String modName = origName;
> modName = modName.replace('\\', replaceChar);
> modName = modName.replace('/', replaceChar);
> modName = modName.replace(':', replaceChar);
> modName = modName.replace('*', replaceChar);
> modName = modName.replace('?', replaceChar);
> modName = modName.replace('"', replaceChar);
> modName = modName.replace('<', replaceChar);
> modName = modName.replace('>', replaceChar);
> modName = modName.replace('|', replaceChar);
>
> FileWriter fWriter = new FileWriter(modName);
>
>
> System.out.println("Writing contents of " + origName + " to "
> +
> "the following file: " + modName);
> for(;;)
> {
> String thisLine = bufRdr.readLine();
> if(thisLine == null)
> break;
>
> fWriter.write(thisLine);
> }
> }
> }
> }
> }
>
> /////////////////// Source code end //////////////////////[/color]


KrOoSh
Guest
 
Posts: n/a
#3: Jul 17 '05

re: Q) Accessing secure websites from a Java application


HttpClient should solve your problem:
http://jakarta.apache.org/commons/ht.../sslguide.html



"Tom N" <tom@nospam.au> wrote in message news:<sCw6c.110468$Wa.32149@news-server.bigpond.net.au>...[color=blue]
> See http://sunsolve.sun.com/pub-cgi/retr...salert%2F57436
>
> The error is because the Class 3 and Class 2 root certificates installed in
> your version of Java have expired. It says you can either download the
> latest JRE or JDK or you can download certificates from verisign and install
> them.
>
> I tried your code and the following is the status of a list of URLs before I
> updated the certificate. I am using JDK 1.4.1_02-b06 by the way. This list
> is taken from the Javaworld article you referenced.
>
> https://www.verisign.com/ failed
> https://happiness.dhs.org/ failed
> https://www.microsoft.com worked
> https://www.sun.com failed (IIRC)
> https://www.ftc.gov failed
>
> Interestingly, those that failed did not all have the same exception thrown.
>
> I updated the cert's as detailed in the sunsolve article and now all of the
> URLs worked except dhs.org, which is because it uses a self-signed
> certifcate (not using a Verisign or similar root certificate).
>
> "Generic Usenet Account" wrote:[color=green]
> > I am trying to create a Java application that reads a list of URLs
> > from a file and stores their contents on the local file system. I
> > have succeeded in accessing normal websites, but I am unable to access
> > the secured websites (using the HTTPS protocol) using this approach.
> >
> > I would greatly appreciate if someone could suggest a way out. I have
> > looked at the HttpsURLConnection class, but unfortunately this class
> > is abstract.
> >
> > [My apolgies to readers of comp.lang.java and comp.sources.d since
> > this posting may appear to be a repost of my previous posting
> > (although it is not) --- Bhat]
> >
> > My source code follows:
> >
> > /////////////////// Source code begin /////////////////////
> >
> > // This program reads a list of URLs to access and store on the local
> > // file system from a file. The name of the file is passed as the
> > // first command line argument. Each URL is on a separate line.
> > // Lines beginning with the '#' character are treated as blanks and
> > // are skipped.
> > //
> > import java.io.*;
> > import java.net.*;
> > import java.security.*;
> >
> >
> > class WebsiteLoader
> > {
> > public static char replaceChar = '~';
> >
> > public static void main(String argv[]) throws IOException
> > {
> > // The following two lines were suggested by the following
> > website:
> > // http://www.javaworld.com/javaworld/j...javatip96.html
> > // They help in suppressing the java.net.MalformedURLException
> > System.setProperty("java.protocol.handler.pkgs",
> > "com.sun.net.ssl.internal.www.protocol");
> > Security.addProvider(new com.sun.net.ssl.internal.ssl.Provider());
> >
> > BufferedReader br;
> > String origName;
> >
> > if(argv.length != 0)
> > {
> > br = new BufferedReader(new FileReader(argv[0]));
> >
> > // Read URLs from the file. Skip blank lines and lines
> > beginning
> > // with the '#' character.
> > for(;;)
> > {
> > origName = br.readLine();
> > if(origName == null)
> > break;
> >
> > origName = origName.trim();
> >
> > if(origName.length() == 0)
> > continue;
> >
> > if(origName.charAt(0) == '#')
> > continue;
> >
> > URL url = new URL(origName);
> > if(url == null)
> > continue;
> >
> > BufferedReader bufRdr = new BufferedReader(new
> > InputStreamReader(url.openStream()));
> >
> > // The name of the file to which the website contents are
> > written
> > // is derived from the URL by substituting the following
> > characters
> > // with some "non-offending" character:
> > // \,/,:,*,?,",<,>,|
> >
> > String modName = origName;
> > modName = modName.replace('\\', replaceChar);
> > modName = modName.replace('/', replaceChar);
> > modName = modName.replace(':', replaceChar);
> > modName = modName.replace('*', replaceChar);
> > modName = modName.replace('?', replaceChar);
> > modName = modName.replace('"', replaceChar);
> > modName = modName.replace('<', replaceChar);
> > modName = modName.replace('>', replaceChar);
> > modName = modName.replace('|', replaceChar);
> >
> > FileWriter fWriter = new FileWriter(modName);
> >
> >
> > System.out.println("Writing contents of " + origName + " to "
> > +
> > "the following file: " + modName);
> > for(;;)
> > {
> > String thisLine = bufRdr.readLine();
> > if(thisLine == null)
> > break;
> >
> > fWriter.write(thisLine);
> > }
> > }
> > }
> > }
> > }
> >
> > /////////////////// Source code end //////////////////////[/color][/color]
Closed Thread