Academic writing needs rewriting and revision multiple times before the final draft is ready. During this process a bunch of textual changes can occur. What is being changed and why it is changed is considered very important factor of a successful argumentative writing. Hence it is important to be able to analyze the revisions and automatically tell if a revised sentence is better or not. However, there is not enough data sets to carry on the experiments. In this work, we have extracted revised sentence pairs from the essay revision corpus and annotated them using Amazon Mechanical Turk. We contribute with a new dataset of revised sentences. We also show results from our sentence level revision quality prediction task.