Download a page in Java when HTTP authentification is required

In your browser, you can type: http://username:password@host.com/securestuff to automatically authenticate if this page is protected. You can’t do this with the URL class, it will come back with an UnknownHostException. But there is a way: create a URLConnection and set the authorization request property (for more information on request properties, check out the link in related links). This example shows you how to retrieve the data on a protected page providing you have the username and password. It makes use of the sun.misc package to do the required Base64 encoding of the concatenated string “username:password”. You may want to replace the use of the “sun.misc” package by your own implementation of Base64 (or another provider) as the availability of sun. classes is not guaranteed in future versions of the JDK.

Main.java:

import sun.misc.*;
import java.net.*;
import java.io.*;
  
public class Main
{
   public static void main(String args[]) 
   {
      try {
         if (args.length != 3) {
            System.err.println("Usage: java Main <URL> <username> <password>");
            System.exit(0);
         }
 
         String authReq = args[1] + ":" + args[2];
         BASE64Encoder B64Encoder = new BASE64Encoder();
         String encodedAuthReq = B64Encoder.encode(authReq.getBytes());
 
         URL url = new URL(args[0]);
         URLConnection connection = url.openConnection();
         connection.setDoInput(true);
         connection.setRequestProperty("Authorization", "basic " + encodedAuthReq);
         connection.connect();
         
         InputStream in = connection.getInputStream();
 
         BufferedReader br = new BufferedReader(new InputStreamReader(in));
   
         String line;
         while ((line = br.readLine()) != null) {
            System.out.println(line);
         }
      }
      catch (MalformedURLException e) {
         System.err.println (e);
      }
      catch (IOException e) {
         System.err.println (e);
      }
   }
}