Trustworthy (news) Project

Current trustworthiness project

Main goal

  • Have an item (webpage, blog posting, tweet, etc) with a unique identifier (url, guid, etc) that has a clam or claims (the lone gunman theory was a hoax!). Be able to associate comments with this item that support or dispute the claim. Comments must point to another item that supports the comment.
  • This new item is now associated via the comment with the original item. Ideally the new item is vetted (say, by using NLP to compare the comment with the supporting item for correlation)
    • Secondary associations such as user, url root, time and place can be linked through these associations as well, but don’t fixate.
  • New items behave just like the original item. They can have comments and links to additional items.
  • Comments pointing at the original item can support or dispute. Comments pointing to supporting links always support.
  • In the naive case, items that have more support than dispute are considered more trustworthy. However, the network can be used. If a source is used to support many comments that dispute an item is itself disputed, that effect should ripple through the network.
  • Users should (at least) be able to look at the network based only on their own comments and on the global comments. It could be possible to define communities as well. The Wikipedia can serve as a model, if we used citations as (supporting?) links, and the preceding text as the comment.

Plan A

  • Pull stories from Google News (top-level feeds: World, national, entertainment, etc.) RSS, parse them, and put them in the database – done
  • Using Alchemy NLP, pull out the authors, subjects, links, etc from the stories. Search for them in the Alchemy News API. Use this to populate relevant tables (author, etc) that can point back to the main article
  • We’ll need some ratings tables as well. The information should include the rating and the links that support the statement. If there is freeform text, we could run some NLP on it for sentiment, etc.
  • Provide the list to the browser as the navigator, with the trustworthiness annotations
  • When a story is clicked, show the associated network with that story. Each item can be clicked to bring up information about that attribute (as a pop-up?)
  • Database thoughts
    • Login and password
    • Sessions should be saveable and shareable
    • Sessions should be able to use or disregard meta information from other users. (Meta information from other does provide another network that can be incorporated?)
    • Information on links can have a variety of flavors
      • Trustworthy (because…)
      • Interesting (short note for why?)
      • Saved (short note for why?)

Plan B:

  • Ok, I’m disappointed with the AlchemyNews API and not sure that the NLP is worth it. We’ll see. In the mean time here’s the new plan
 "relevance": "0.939823",
 "sentiment": {
 "score": "-0.470436",
 "type": "negative"
 "text": "chattanooga gunman"
 "relevance": "0.734766",
 "sentiment": {
 "score": "-0.570721",
 "type": "negative"
 "text": "chattanooga shooter"
 "relevance": "0.706188",
 "sentiment": {
 "mixed": "1",
 "score": "-0.0326505",
 "type": "negative"
 "text": "chattanooga shooting"
  • That might be a nice way to combine everything. And don’t forget about tracking individual ratings…

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s