Following training, we chose three evaluation metrics; closeness, directional accuracy, and a simulated trading engine. The closeness metric evaluated the difference between the predicted value and the actual stock price, measured using mean squared error (MSE). Directional accuracy measured the up/down direction of the predicted stock price compared with the actual direction of the stock price. While the inclusion of directional accuracy may not seem intuitive given the measure of closeness, it is possible to be close in prediction yet predict the wrong direction of movement. This leads us to a third evaluation measure using a simulated trading engine that invests $1,000 per trade and follows simple trading rules. The rules implemented by our trading engine are a modified version of those proposed by Mittermayer [2004] to maximize short-term trading profit. Our simulated trading engine evaluates each news article and will buy/short the stock if the predicted +20 minute stock price is greater than or equal to 1% movement from the stock price at the time the article was released. Any bought/shorted stocks are then sold after 20 minutes. This assumes a zero transaction cost which is consistent with the research of Lavrenko [Lavrenko et al. 2000a, 2000b] and Mittermayer [2004] who argue that trading in volume will offset the costs of trading.