Tokenizing a text into sentences

Use the BreakIterator class. First get an instance through the factory method getSentenceInstance, initialize it with the text using setText and iterate through the elements with first and next.

This example uses the following text: text.txt:

It is, in fact, nothing short of a miracle that the modern methods 
of instruction have not yet entirely strangled the holy curiosity 
of inquiry; for this delicate little plant, aside from stimulation, 
stands mainly in need of freedom; without this it goes to wrack and 
ruin without fail. It is a very grave mistake to think that the 
enjoyment of seeing and searching can promoted by means of coercion 
and sense of duty. To the contrary, I believe that it would be 
possible to rob even a healthy beast of prey of its voraciousness, 
if it were possible, with the aid of a whip, to force the beast to 
devour continuously, even when not hungry,especiallyif the food, 
handed out under such coercion, were to be selected accordingly. 

Albert Einstein 

Main.java:

import java.text.*;
import java.io.*;
  
public class Main {
   public static void main(String args[]) {
      if (args.length != 1) {
         System.err.println("Usage: java Main <file.txt>");
         System.exit(1);
      }
 
      String text = readFile(args[0]);
 
      BreakIterator biSentence = BreakIterator.getSentenceInstance();
      biSentence.setText(text);
      System.out.println("Splitting text up into sentencesn");
      printInstances(biSentence, text);
   }  
 
   public static void printInstances(BreakIterator bi, String text) {
      int start = bi.first();
      int end = bi.next();
      while (end != BreakIterator.DONE) {
         System.out.println("Found: " + text.substring(start, end) + "n");
         start = end;
         end = bi.next();
      }
   }
 
   public static String readFile(String filename) {
      StringBuffer total = new StringBuffer();
 
      try {
         BufferedReader br = new BufferedReader(new FileReader(filename));
         String line;
         while ((line = br.readLine()) != null) {
            total.append(line + "n");
         }
      }
      catch(Exception e) {
         e.printStackTrace();
         System.exit(1);
      }
      return total.toString();
   }
}

prints out:

Splitting text up into sentences

Found: It is, in fact, nothing short of a miracle that the modern methods 
of instruction have not yet entirely strangled the holy curiosity 
of inquiry; for this delicate little plant, aside from stimulation, 
stands mainly in need of freedom; without this it goes to wrack and 
ruin without fail. 

Found: It is a very grave mistake to think that the 
enjoyment of seeing and searching can promoted by means of coercion 
and sense of duty. 

Found: To the contrary, I believe that it would be 
possible to rob even a healthy beast of prey of its voraciousness, 
if it were possible, with the aid of a whip, to force the beast to 
devour continuously, even when not hungry,especiallyif the food, 
handed out under such coercion, were to be selected accordingly. 



Found: Albert Einstein

Note that without a semantic analyser, there will be cases where the breaking up in sentences will result in erronous results, especially when dealing with decimal numbers in the text or a phrase like “Dr. Jekyll & Mr. Hide” where the “.” does not represent the end of a sentence.