By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
424,972 Members | 901 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 424,972 IT Pros & Developers. It's quick & easy.

Q) Accessing secure websites from a Java application

P: n/a
I am trying to create a Java application that reads a list of URLs
from a file and stores their contents on the local file system. I
have succeeded in accessing normal websites, but I am unable to access
the secured websites (using the HTTPS protocol) using this approach.

I would greatly appreciate if someone could suggest a way out. I have
looked at the HttpsURLConnection class, but unfortunately this class
is abstract.

[My apolgies to readers of comp.lang.java and comp.sources.d since
this posting may appear to be a repost of my previous posting
(although it is not) --- Bhat]

My source code follows:

/////////////////// Source code begin /////////////////////

// This program reads a list of URLs to access and store on the local
// file system from a file. The name of the file is passed as the
// first command line argument. Each URL is on a separate line.
// Lines beginning with the '#' character are treated as blanks and
// are skipped.
//
import java.io.*;
import java.net.*;
import java.security.*;
class WebsiteLoader
{
public static char replaceChar = '~';

public static void main(String argv[]) throws IOException
{
// The following two lines were suggested by the following
website:
// http://www.javaworld.com/javaworld/j...javatip96.html
// They help in suppressing the java.net.MalformedURLException
System.setProperty("java.protocol.handler.pkgs",
"com.sun.net.ssl.internal.www.protocol");
Security.addProvider(new com.sun.net.ssl.internal.ssl.Provider());

BufferedReader br;
String origName;

if(argv.length != 0)
{
br = new BufferedReader(new FileReader(argv[0]));

// Read URLs from the file. Skip blank lines and lines
beginning
// with the '#' character.
for(;;)
{
origName = br.readLine();
if(origName == null)
break;

origName = origName.trim();

if(origName.length() == 0)
continue;

if(origName.charAt(0) == '#')
continue;

URL url = new URL(origName);
if(url == null)
continue;

BufferedReader bufRdr = new BufferedReader(new
InputStreamReader(url.openStream()));

// The name of the file to which the website contents are
written
// is derived from the URL by substituting the following
characters
// with some "non-offending" character:
// \,/,:,*,?,",<,>,|

String modName = origName;
modName = modName.replace('\\', replaceChar);
modName = modName.replace('/', replaceChar);
modName = modName.replace(':', replaceChar);
modName = modName.replace('*', replaceChar);
modName = modName.replace('?', replaceChar);
modName = modName.replace('"', replaceChar);
modName = modName.replace('<', replaceChar);
modName = modName.replace('>', replaceChar);
modName = modName.replace('|', replaceChar);

FileWriter fWriter = new FileWriter(modName);
System.out.println("Writing contents of " + origName + " to "
+
"the following file: " + modName);
for(;;)
{
String thisLine = bufRdr.readLine();
if(thisLine == null)
break;

fWriter.write(thisLine);
}
}
}
}
}

/////////////////// Source code end //////////////////////
Jul 17 '05 #1
Share this Question
Share on Google+
2 Replies


P: n/a
See http://sunsolve.sun.com/pub-cgi/retr...salert%2F57436

The error is because the Class 3 and Class 2 root certificates installed in
your version of Java have expired. It says you can either download the
latest JRE or JDK or you can download certificates from verisign and install
them.

I tried your code and the following is the status of a list of URLs before I
updated the certificate. I am using JDK 1.4.1_02-b06 by the way. This list
is taken from the Javaworld article you referenced.

https://www.verisign.com/ failed
https://happiness.dhs.org/ failed
https://www.microsoft.com worked
https://www.sun.com failed (IIRC)
https://www.ftc.gov failed

Interestingly, those that failed did not all have the same exception thrown.

I updated the cert's as detailed in the sunsolve article and now all of the
URLs worked except dhs.org, which is because it uses a self-signed
certifcate (not using a Verisign or similar root certificate).

"Generic Usenet Account" wrote:
I am trying to create a Java application that reads a list of URLs
from a file and stores their contents on the local file system. I
have succeeded in accessing normal websites, but I am unable to access
the secured websites (using the HTTPS protocol) using this approach.

I would greatly appreciate if someone could suggest a way out. I have
looked at the HttpsURLConnection class, but unfortunately this class
is abstract.

[My apolgies to readers of comp.lang.java and comp.sources.d since
this posting may appear to be a repost of my previous posting
(although it is not) --- Bhat]

My source code follows:

/////////////////// Source code begin /////////////////////

// This program reads a list of URLs to access and store on the local
// file system from a file. The name of the file is passed as the
// first command line argument. Each URL is on a separate line.
// Lines beginning with the '#' character are treated as blanks and
// are skipped.
//
import java.io.*;
import java.net.*;
import java.security.*;
class WebsiteLoader
{
public static char replaceChar = '~';

public static void main(String argv[]) throws IOException
{
// The following two lines were suggested by the following
website:
// http://www.javaworld.com/javaworld/j...javatip96.html
// They help in suppressing the java.net.MalformedURLException
System.setProperty("java.protocol.handler.pkgs",
"com.sun.net.ssl.internal.www.protocol");
Security.addProvider(new com.sun.net.ssl.internal.ssl.Provider());

BufferedReader br;
String origName;

if(argv.length != 0)
{
br = new BufferedReader(new FileReader(argv[0]));

// Read URLs from the file. Skip blank lines and lines
beginning
// with the '#' character.
for(;;)
{
origName = br.readLine();
if(origName == null)
break;

origName = origName.trim();

if(origName.length() == 0)
continue;

if(origName.charAt(0) == '#')
continue;

URL url = new URL(origName);
if(url == null)
continue;

BufferedReader bufRdr = new BufferedReader(new
InputStreamReader(url.openStream()));

// The name of the file to which the website contents are
written
// is derived from the URL by substituting the following
characters
// with some "non-offending" character:
// \,/,:,*,?,",<,>,|

String modName = origName;
modName = modName.replace('\\', replaceChar);
modName = modName.replace('/', replaceChar);
modName = modName.replace(':', replaceChar);
modName = modName.replace('*', replaceChar);
modName = modName.replace('?', replaceChar);
modName = modName.replace('"', replaceChar);
modName = modName.replace('<', replaceChar);
modName = modName.replace('>', replaceChar);
modName = modName.replace('|', replaceChar);

FileWriter fWriter = new FileWriter(modName);
System.out.println("Writing contents of " + origName + " to "
+
"the following file: " + modName);
for(;;)
{
String thisLine = bufRdr.readLine();
if(thisLine == null)
break;

fWriter.write(thisLine);
}
}
}
}
}

/////////////////// Source code end //////////////////////

Jul 17 '05 #2

P: n/a
HttpClient should solve your problem:
http://jakarta.apache.org/commons/ht.../sslguide.html

"Tom N" <to*@nospam.au> wrote in message news:<sC*******************@news-server.bigpond.net.au>...
See http://sunsolve.sun.com/pub-cgi/retr...salert%2F57436

The error is because the Class 3 and Class 2 root certificates installed in
your version of Java have expired. It says you can either download the
latest JRE or JDK or you can download certificates from verisign and install
them.

I tried your code and the following is the status of a list of URLs before I
updated the certificate. I am using JDK 1.4.1_02-b06 by the way. This list
is taken from the Javaworld article you referenced.

https://www.verisign.com/ failed
https://happiness.dhs.org/ failed
https://www.microsoft.com worked
https://www.sun.com failed (IIRC)
https://www.ftc.gov failed

Interestingly, those that failed did not all have the same exception thrown.

I updated the cert's as detailed in the sunsolve article and now all of the
URLs worked except dhs.org, which is because it uses a self-signed
certifcate (not using a Verisign or similar root certificate).

"Generic Usenet Account" wrote:
I am trying to create a Java application that reads a list of URLs
from a file and stores their contents on the local file system. I
have succeeded in accessing normal websites, but I am unable to access
the secured websites (using the HTTPS protocol) using this approach.

I would greatly appreciate if someone could suggest a way out. I have
looked at the HttpsURLConnection class, but unfortunately this class
is abstract.

[My apolgies to readers of comp.lang.java and comp.sources.d since
this posting may appear to be a repost of my previous posting
(although it is not) --- Bhat]

My source code follows:

/////////////////// Source code begin /////////////////////

// This program reads a list of URLs to access and store on the local
// file system from a file. The name of the file is passed as the
// first command line argument. Each URL is on a separate line.
// Lines beginning with the '#' character are treated as blanks and
// are skipped.
//
import java.io.*;
import java.net.*;
import java.security.*;
class WebsiteLoader
{
public static char replaceChar = '~';

public static void main(String argv[]) throws IOException
{
// The following two lines were suggested by the following
website:
// http://www.javaworld.com/javaworld/j...javatip96.html
// They help in suppressing the java.net.MalformedURLException
System.setProperty("java.protocol.handler.pkgs",
"com.sun.net.ssl.internal.www.protocol");
Security.addProvider(new com.sun.net.ssl.internal.ssl.Provider());

BufferedReader br;
String origName;

if(argv.length != 0)
{
br = new BufferedReader(new FileReader(argv[0]));

// Read URLs from the file. Skip blank lines and lines
beginning
// with the '#' character.
for(;;)
{
origName = br.readLine();
if(origName == null)
break;

origName = origName.trim();

if(origName.length() == 0)
continue;

if(origName.charAt(0) == '#')
continue;

URL url = new URL(origName);
if(url == null)
continue;

BufferedReader bufRdr = new BufferedReader(new
InputStreamReader(url.openStream()));

// The name of the file to which the website contents are
written
// is derived from the URL by substituting the following
characters
// with some "non-offending" character:
// \,/,:,*,?,",<,>,|

String modName = origName;
modName = modName.replace('\\', replaceChar);
modName = modName.replace('/', replaceChar);
modName = modName.replace(':', replaceChar);
modName = modName.replace('*', replaceChar);
modName = modName.replace('?', replaceChar);
modName = modName.replace('"', replaceChar);
modName = modName.replace('<', replaceChar);
modName = modName.replace('>', replaceChar);
modName = modName.replace('|', replaceChar);

FileWriter fWriter = new FileWriter(modName);
System.out.println("Writing contents of " + origName + " to "
+
"the following file: " + modName);
for(;;)
{
String thisLine = bufRdr.readLine();
if(thisLine == null)
break;

fWriter.write(thisLine);
}
}
}
}
}

/////////////////// Source code end //////////////////////

Jul 17 '05 #3

This discussion thread is closed

Replies have been disabled for this discussion.