Generating a MAC that uses SHA-1

The following example consists of a Sender and a Receiver.

- The Sender creates a message, generates a secret key, generates a mac given the message and the key and writes both the message and the mac to the file “message.ser”. Typically, this would be send to the other party.
A MAC works with a secret key. The weak point is that both the sender and the receiver must know the secret key. In the example, the secret key is stored in a keystore “keystore.kst” in the current directory.

- The Receiver reads “message.ser” and extracts the message and the MAC. To find out whether the message was tampered with, it calculates the MAC again using the message and the secret key that it reads out from “keystore.kst”.

Note that to use the keystore for secret keys, the keystore algorithm “JCEKS” must be used. “JCEKS” is implemented by the security provider that comes with JCE. You can download it here.

I added jce1_2_2.jar and sunjce_provider to my classpath. The code is documented.

Sender.java:

import javax.crypto.spec.*;
import javax.crypto.Mac;
import java.security.*;
import javax.crypto.*;
import sun.misc.*;
import java.io.*;
 
public class Sender
{
   public static void main(String []args) throws Exception {
      // message to send 
      String message = "the sun is green and the grass shines";
 
      // generate a random key and store it in keystore.kst in current directory 
      SecretKey key = generateAndStoreKey();
 
      // generate a MAC for the message, given the secret key
      byte[] mac = generateMAC(message, key);
      
      System.out.println("HMAC-MD5 for message '" + message + "':");
      System.out.println("t" + new BASE64Encoder().encode(mac));
      System.out.println("Writing message+mac to file 'message.ser'");
      
      // writing both the message and the mac to message.ser
      ObjectOutputStream oos = new ObjectOutputStream(new FileOutputStream("message.ser"));
      oos.writeObject(message);
      oos.writeObject(mac);
      oos.close();
   }
 
   public static SecretKey generateAndStoreKey() throws Exception {
      KeyStore ks = KeyStore.getInstance("JCEKS");
      // initiaze new KeyStore, must do!
      ks.load(null, null);
 
      // generate random key
      SecureRandom sr = new SecureRandom();
      byte[] b = new byte[20];
      sr.nextBytes(b);
      SecretKey key = new SecretKeySpec(b, "HmacSHA1");
 
      // save secret key in our new keystore with alias MySecretKey
      FileOutputStream fos = new FileOutputStream("keystore.kst");
      ks.setKeyEntry("MySecretKey", key, "123456".toCharArray(), null);
      ks.store(fos, "123456".toCharArray());
      fos.close();
 
      return key;
   }
 
   public static byte[] generateMAC(String message, SecretKey key) throws Exception {
      // generate message digest based on key
      Mac mac = Mac.getInstance("HmacMD5");
      mac.init(key);
 
      return mac.doFinal(message.getBytes());
   }
}

Receiver.java:

import javax.crypto.spec.*;
import javax.crypto.Mac;
import java.security.*;
import javax.crypto.*;
import sun.misc.*;
import java.io.*;
 
public class Receiver
{
   public static void main(String []args) throws Exception {
      // read in message and mac from message.ser
      ObjectInputStream ois = new ObjectInputStream(new FileInputStream("message.ser"));
      String message = (String) ois.readObject();
      byte[] mac = (byte[]) ois.readObject();
 
      System.out.println("Original message: " + message);
 
      // get secret key from keystore keystore.kst
      SecretKey key = getSecretKey();
 
      // generate mac again given this key
      byte[] newMac = getMAC(message, key);
 
      // ensure the new mac is equal to the one that was "sent" to us
      if (MessageDigest.isEqual(mac, newMac)) {
         System.out.println("Message was not tampered with!");
      }
      else {
         System.out.println("Message has been tampered with!");
      }
   }
 
   public static byte[] getMAC(String message, SecretKey key) throws Exception {
      // generate message digest based on key
      Mac mac = Mac.getInstance("HmacMD5");
      mac.init(key);
  
      return mac.doFinal(message.getBytes());
   }

   public static SecretKey getSecretKey() throws Exception {
      FileInputStream fis = new FileInputStream("keystore.kst");
      KeyStore ks = KeyStore.getInstance("JCEKS");
      ks.load(fis, "123456".toCharArray());
 
      SecretKey key = (SecretKey) ks.getKey("MySecretKey", "123456".toCharArray());
  
      return key;
   }
}

sample run:

C:mybouncycastlemacsha1>java Sender
HMAC-MD5 for message 'the sun is green and the grass shines':
        XmvNl18rRj9ql4JkS3N8vw==
Writing message+mac to file 'message.ser'
 
C:mybouncycastlemacsha1>java Receiver
Original message: the sun is green and the grass shines
Message was not tampered with!

What is hybrid cryptography?

Symmetric and asymmetric ciphers each have their own advantages and disadvantages. Symmetric ciphers are significantly faster (Schneier states “at least 1000 times faster”) than asymmetric ciphers, but require all parties to somehow share a secret (the key). The asymmetric algorithms allow public key infrastructures and key exchange systems, but at the cost of speed.

A hybrid cryptosystem is a protocol using multiple ciphers of different types together, each to it’s best advantage. One common approach is to generate a random secret key for a symmetric cipher, and then encrypt this key via an asymmentric cipher using the recipient’s public key. The message itself is then encrypted using the symmetric cipher and the secret key. Both the encrypted secret key and the encrypted message are then sent to the recipient.

The recipient decrypts the secret key first, using his/her own private key, and then uses that key to decrypt the message. This is basically the approach used in PGP.

Debugging permissions set by policy files

You can use the option

   -Djava.security.debug=<PARAM>
 
where <PARAM> is one of the following
   all       (turn on all debugging)
   access    (print all checkPermission results)
   jar       (jar verification)
   policy    (loading and granting)
   scl	 (permission SecureClassLoader assigns) 

The following can be used with access (for example access.domain):

   stack	 (include stack trace) 
   domain	 (dumps all domains in context)
   failure   (before throwing exception, dump stack
              and domain that didn't have permission)

When you’re testing policy problems with the Java plug-in, you can go the the plug-in control panel and to Java Runtime Parameters:

   -Djava.security.debug=<PARAM>

However, when I tried this out (with Java plug-in) it crashed the browser (IE5.5) on several params. The parameter policy worked and gave me sufficient results to solve my problem.

Encrypting/decrypting a message using a symmetric encryption method (eg. DES)

Eric Young has written an implementation of DES. Here are the source files:

Des.java
DesCrypt.java
DesKey.java
Int32Manipulator.java

The Des.java file has been slightly modified (added a memset method to ensure 8 bytes are used for the key).

Here’s an example on how to use these classes:

public class Main
{
   public static void main(String []args) {
      // key (8 bytes) 
      String key = "12345678";
 
      // encrypt using Eric Young's DES Java implementation
      byte[] de = Des.encrypt(key, "Esus.com, aiming to provide the largest Java directory on the web");
 
      // decrypt 
      String s = Des.decrypt(key, de);
      System.out.println(s);
   }
}

Encrypting/decrypting a message using a hybrid encryption method

The following code demonstrates using a hybrid encryption method to send a message from a producer to a consumer. This example uses RSA for asymmetric encryption and AES (Rijndael) for symmetric encryption.

The process on the producer side is as follows:

  1. Generate a random symmetric key that will be used
    for this message only.

  2. Encrypt the message to be sent using the symmetric algorithm
    and the random key.

  3. Encrypt the random key using the asymmetric algorithm and
    the consumer’s public key.

  4. Send both the encrypted key and the encrypted message to
    the consumer.

At this point the producer can erase the random symmetric key; it will never be needed again. The ability to utilize a session-only secret key is an important advantage of this technique.

The consumer first decrypts the asymmetric key using his/her private key,
and then uses that key to decrypt the message itself.

HybridTest.java:

import org.bouncycastle.jce.provider.BouncyCastleProvider;
import org.bouncycastle.util.encoders.* ;
  
import java.security.*;
import javax.crypto.*;
import javax.crypto.spec.* ;
 
public class HybridTest {
 
    // asymmetric algorithms used
    public static String asymKeyAlgorithm = "RSA" ;
    public static String asymAlgorithm = "RSA/OAEP" ;
    public static int asymKeyAlgorithmStrength = 1024 ;
    public static String signatureAlgorithm = "SHA1WithRSAEncryption" ;
 
    // symmetric algorithms used
    public static String symKeyAlgorithm = "RIJNDAEL" ;
    public static String symAlgorithm = "RIJNDAEL" ;
    //public static String symAlgorithm = "BLOWFISH" ;
    //public static String symAlgorithm = "TWOFISH" ;
    public static int symAlgorithmStrength = 256 ;
 
    static public void main(String[] args) {
 
	String testData = "You can't have your kayak and heat it, too." ;
 
	Base64 b64 = new Base64() ;
 
	try {
	    // make sure the BC provider is registered.
	    Security.addProvider(new BouncyCastleProvider());
 
	    SecureRandom sr = new SecureRandom() ;
 
	    System.out.println("Original message byte count: " + testData.getBytes().length) ;
 
    /***
     * Generate consumer keys for test purposes.  In Real Life(TM) the
     * producer would need to know only the consumer's public key.
     ***/
	    KeyPairGenerator gen = KeyPairGenerator.getInstance(asymKeyAlgorithm, "BC");
	    gen.initialize(asymKeyAlgorithmStrength, sr);

	    System.out.println("Generating key . . .") ;
	    KeyPair consumerPair = gen.generateKeyPair();
 
    /***
     * on the producer side:
     * 1. Generate a secret key.
     * 2. Use asymmetric algorithm to encrypt the secret key for consumer
     * 3. Use symmetric algorithm to encrypt message using the secret key
     ***/
	    // generate a random secret key
	    KeyGenerator kg = KeyGenerator.getInstance(symKeyAlgorithm) ;
	    kg.init(symAlgorithmStrength, sr) ;
	    SecretKey cipherKey = kg.generateKey() ;
	    System.out.println("Generated cipher key, proceeding: " + cipherKey.getAlgorithm()) ;
  
	    // encrypt the secret key using the consumer's public key
	    byte[] encryptedSecretKey = encrypt(cipherKey.getEncoded(), consumerPair.getPublic()) ;
  
	    // encrypt the testData using the secret key
	    byte[] encryptedData = encrypt(testData.getBytes(), cipherKey) ;
 	    
	    System.out.println("Encrypted byte count: " + encryptedData.length) ;
	    System.out.println("Encrypted message: [" + new String(b64.encode(encryptedData)) + "]") ;
 
    /***
     * uncomment to befoul the encrypted data for testing purposes
     ***/
	    //encryptedData[8] = 0 ;
 	    
    /***
     * and now on the consumer side:
     * 1. Use asymmetric algorithm and consumer's private key to decrypt the secret key
     * 2. Use symmetric algorithm and secret key to decrypt message.
     ***/
	    // first get the secret key back with the consumer's private key
	    byte[] encodedSecretKey = decrypt(encryptedSecretKey, consumerPair.getPrivate()) ;
	    SecretKey sKey = new SecretKeySpec(encodedSecretKey, symAlgorithm) ;
	    System.out.println("Secret key decoded.") ;
 
	    // decrypt the message using the secret key
	    byte[] clearData = decrypt(encryptedData, sKey) ;
	    System.out.println("Data decoded, byte count: " + clearData.length) ;
	    System.out.println("Decrypted message: [" + new String(clearData) + "]") ;
	    }
	catch ( Exception ex ) {
	    ex.printStackTrace() ;
	    }
 	    
	System.exit(0);
	}
 
    public static byte[] encrypt(byte[] toEncrypt, SecretKey key)
				    throws GeneralSecurityException {
 
	Cipher cipher = Cipher.getInstance(symAlgorithm) ;
	System.out.println("got cipher, blocksize = " + cipher.getBlockSize()) ;
	cipher.init(Cipher.ENCRYPT_MODE, key) ;
 
	byte[] result = cipher.doFinal(toEncrypt) ;
	return result ;
	}
 
    public static byte[] encrypt(byte[] toEncrypt, PublicKey key)
				    throws GeneralSecurityException {

	Cipher cipher = Cipher.getInstance(asymAlgorithm) ;
	cipher.init(Cipher.ENCRYPT_MODE, key) ;
 
	byte[] result = cipher.doFinal(toEncrypt) ;
	return result ;
	}
 
    public static byte[] decrypt(byte[] toDecrypt, SecretKey key)
				    throws GeneralSecurityException {
 
	Cipher deCipher = Cipher.getInstance(symAlgorithm) ;
	deCipher.init(Cipher.DECRYPT_MODE, key) ;
 
	byte[] result = deCipher.doFinal(toDecrypt) ;
	return result ;
	}
 
    public static byte[] decrypt(byte[] toDecrypt, PrivateKey key)
				    throws GeneralSecurityException {
 
	Cipher deCipher = Cipher.getInstance(asymAlgorithm) ;
	deCipher.init(Cipher.DECRYPT_MODE, key) ;
 
	byte[] result = deCipher.doFinal(toDecrypt) ;
	return result ;
	}
    }

(tested with the BouncyCastle JCE provider, http://www.bouncycastle.org)

What is a digital signature?

A digital signature is an extra chunk of data send along with an (encrypted) message which tells the receiver that the message is coming from the one who claims to have sent it. It ensures that the message has not been tampered with by a man-in-the-middle, a person who spies on the communication line and intercepts messages. You also have a proof that the sender is the one who holds the private key of a public-private key pair.

It goes as follows:

   - Two parties, A and B.  A wants to send a message M to B.
  
      +-----+                              +-----+
      |  A  |                              |  B  |
      +-----+                              +-----+
 - holds private key PVT of       - holds public key PUB of
   public-private key pair          public-private key pair
 
Sender:
   1. A calculates a message digest MD on M
   2. A encrypts that message digest MD with his private
      key PVT (A is the only one that has the private key
      that belongs to him!) = digital signature DS 
   3. A sends message M and DS to B
 
Receiver:
   1. B receives the message M and digital signature DS
   2. B decrypts the digital signature DS with the public
      key PUB and gets MD
   3. B calculates a message digest on M = MD2
   4. B compares MD with MD2.  Equality means that the
      sender must have had access to the private key of 
      the PVT-PUB key pair.

But who says this public/private pair belongs to me? I could have identified myself as someone else, generate such a pair and distribute the public key to a sender who believes I am someone else and trick him with this above process claiming to be that someone else…

That’s where digital certificates come in. Check out What is a digital certificate?

What is symmetric cryptography?

In Symmetric Cryptography (also: secret key cryptography), an algorithm is used to scramble the message using a secret key in such a way that it becomes unusable to all except the ones that have access to that secret key. The most widely known symmetric cryptographic algorithm is DES, developed by IBM in the 70ies. It uses a key of 56 bits and operates on chunks of 64 bits at a time. The problem with DES is the short key length, making it possible for someone, that doesn’t have access to the private key, to try all possible keys. DESede provides a significant improvement with the key being 3 times as large.

Symmetric cryptography works very well if encryption and decryption are eventually performed by the same party. But if there are 2 or more parties involved, there is the question of how to exchange the secret key without anyone spying. Look at asymettric cryptography for a solution.

Downloading a secure page (HTTPS) with Java

If you try to get data from an HTTPS-enabled page, you will get a MalformedURLException, as shown in following example:

Main.java:

import java.net.*;
import java.io.*;

public class Main {
   public static void main(String[] args) throws Exception {
      URL url = new URL("https://www.sun.com");
      BufferedReader br = new BufferedReader(new InputStreamReader(url.openStream()));
      String line;

      while ((line = br.readLine()) != null) {
         System.out.println(line);
      }
      br.close();
   }
}

outputs:

C:>java Main
Exception in thread "main" java.net.MalformedURLException: unknown protocol: https
        at java.net.URL.<init>(URL.java:497)
        at java.net.URL.<init>(URL.java:364)
        at java.net.URL.<init>(URL.java:308)
        at Main.main(Main.java, Compiled Code)

This is because Sun has no implementation for the HTTPS protocol in their core libraries. However, they created an reference implementation called Java Secure Socket Extension (JSSE) available as seperate download at http://java.sun.com/products/jsse. Include jcert.jar, jnet.jar and jsse.jar in your classpath and add the following two lines of code to the previous program:

      System.setProperty("java.protocol.handler.pkgs", "com.sun.net.ssl.internal.www.protocol");
      Security.addProvider(new com.sun.net.ssl.internal.ssl.Provider());

Here’s the updated example that works:

import java.security.*;
import java.net.*;
import java.io.*;

public class Main {
   public static void main(String[] args) throws Exception {
      // create a new protocol handler
      System.setProperty("java.protocol.handler.pkgs", "com.sun.net.ssl.internal.www.protocol");

      // protocol handler uses this security provider
      Security.addProvider(new com.sun.net.ssl.internal.ssl.Provider());
      URL url = new URL("https://www.sun.com");
      BufferedReader br = new BufferedReader(new InputStreamReader(url.openStream()));
      String line;

      while ((line = br.readLine()) != null) {
         System.out.println(line);
      }

      br.close();
   }
}

What is a digital certificate?

First read the question What is a digital signature? You’ll then understand what the problem is with those signatures. You can’t really proof that the public key that the sender owns to decrypt the message actually belongs to you. That’s where a digital certificate comes in. Those certificates are issued by a certificate authority (CA) which you would have to pay and give lots of information about your identity. You’d also have to send them your public key. The CA uses all this information to generate a certificate and send it back to you. The standard for digital certificates is X.509. It contains information about your identity, your public key and things like a validity period, etc. It’s important that you choose a well-known CA that other people will trust. The CA will have encrypted the certificate with their private key.

When party A now sends a message to party B, it will sign the message with his certificate. Party B will ensure this certificate is correct by decrypting it using the CA’s public key. Now party B has access to your public key and can decrypt the signature to verify the authentication.

BTW, you can also generate a certificate yourself (eg. using the tool keytool) and use that. But people are not likely to accept that certificate as there is no trusted third party involved.

The standard for digital certificates is X.509. This standard allows for certificate chaining. This allows certificates to be put in a hierarchy. Maybe party B doesn’t trust the first CA, but goes up in the hierarchy and finds one that he trusts. For example, an employee of a company might get a certificate from his company who is in turned signed by Thawte (the root CA). Now suppose that employee uses his certificate to try to get the receiver trust him. The receiver isn’t eager to trust the one that issued the certificate, but since that certificate is signed in turn by Thawte, he trust the original certificate.

Cipher modes for symmetric encryption

  • Electronic Code Book Mode (ECB) (http://www.rsasecurity.com/rsalabs/faq/2-1-4-2.html)
    This mode encrypts all plaintext blocks independently. This means that two identical plaintext blocks are encrypted the same way. It’s a very fast mode (encryption can be done in parallel) but it is not as secure because certain patterns can be discovered in the resulting encrypted text. For example, I encrypted the message “aaaaaaaaeeeeeeeeaaaaaaaaaaaaaaaa” with DES/ECB. This is the encrypted text in an array of byte values:
    -74 -36 103 -124 95 62 54 -117
    70 -70 -61 -9 17 95 54 -114
    -74 -36 103 -124 95 62 54 -117
    -74 -36 103 -124 95 62 54 -117
    -88 109 -8 8 -14 -108 19 122 
  • Cipher Block Chaining Mode (CBC) ( http://www.rsasecurity.com/rsalabs/faq/2-1-4-3.html)
    This mode solves the problem of ECB by XOR’ring the result of the encryption of the previous block with the next block before encrypting the next block. It’s slower as it cannot be done in parallel. The method also uses an initialization vector (IV) as a seed. The first plaintext block is XOR’red with that seed. This ensures that two identical plaintext messages are not encrypted identically.
  • Cipher Feedback Mode (CFB) (http://www.rsasecurity.com/rsalabs/faq/2-1-4-4.html)
    This mode is similar to CBC except that it allows for processing smaller increments of plaintext instead of an entire block. It uses a “register” that has the size of a block and initally contains the seed (initialization vector). Everytime x number of bits are to be encrypted, they are XOR’red with the leftmost x bits in the register. The register is shifted left by x bits and the encrypted bits are appended to it. Since you can encrypt bytes at a time, it is useful for transmission of small chunks of data, eg. chat. But incorrectly transmitting one bit would result in a propagation of the error.
  • Output Feedback Mode (OFB) ( http://www.rsasecurity.com/rsalabs/faq/2-1-4-5.html )
    This mode is similar to CFB except that bit errors that occur during transmission are not propagated to affect the decryption of subsequent blocks.