Sentiment analysis with NLTK python for sentences using sample data or webservice?

Posted by Ke on Stack Overflow See other posts from Stack Overflow or by Ke
Published on 2010-05-14T07:04:19Z Indexed on 2010/05/28 8:11 UTC
Read the original article Hit count: 637

Filed under:
|
|
|

I am embarking upon a NLP project for sentiment analysis.

I have successfully installed NLTK for python (seems like a great piece of software for this). However,I am having trouble understanding how it can be used to accomplish my task.

Here is my task:

  1. I start with one long piece of data (lets say several hundred tweets on the subject of the UK election from their webservice)
  2. I would like to break this up into sentences (or info no longer than 100 or so chars) (I guess i can just do this in python??)
  3. Then to search through all the sentences for specific instances within that sentence e.g. "David Cameron"
  4. Then I would like to check for positive/negative sentiment in each sentence and count them accordingly

NB: I am not really worried too much about accuracy because my data sets are large and also not worried too much about sarcasm.

Here are the troubles I am having:

  1. All the data sets I can find e.g. the corpus movie review data that comes with NLTK arent in webservice format. It looks like this has had some processing done already. As far as I can see the processing (by stanford) was done with WEKA. Is it not possible for NLTK to do all this on its own? Here all the data sets have already been organised into positive/negative already e.g. polarity dataset http://www.cs.cornell.edu/People/pabo/movie-review-data/ How is this done? (to organise the sentences by sentiment, is it definitely WEKA? or something else?)

  2. I am not sure I understand why WEKA and NLTK would be used together. Seems like they do much the same thing. If im processing the data with WEKA first to find sentiment why would I need NLTK? Is it possible to explain why this might be necessary?

I have found a few scripts that get somewhat near this task, but all are using the same pre-processed data. Is it not possible to process this data myself to find sentiment in sentences rather than using the data samples given in the link?

Any help is much appreciated and will save me much hair!

Cheers Ke

© Stack Overflow or respective owner

Related posts about nlp

Related posts about nltk