How An Algorithim Caught The Ebola Outbreak

Science: this is why we keep you around:

Nine days before the World Health Organization announced the African Ebola outbreak now making headlines, an algorithm had already spotted it. HealthMap, a data-driven mapping tool developed out of Boston Children’s Hospital, detected a “mystery hemorrhagic fever” after mining thousands of web-based data sources for clues.

“We’ve been operating HealthMap for over eight years now,” says cofounder Clark Freifeld. “One of the main things that has allowed it to flourish is the availability of large amounts of public event data being accessible on the Internet.”

[…]

As anyone who’s ever looked at the Internet knows, any bulk consumption of web content is bound to scoop up tons of noise, especially when sources like Twitter and blogs are involved. To cope with this, HealthMap applies a machine learning algorithm to filter out irrelevant information like posts about “Bieber fever” or uses of terms like “infection” and “outbreak” that don’t pertain to actual public health events.

“The algorithm actually looks at hundreds of thousands of example articles that have been labeled by our analysts and uses the examples to pick up on key words and phrases that tend to be associated with actual outbreak reports,” explains Freifeld. “The algorithm is continually improving, learning from our analysts through a feedback loop.”

Advertisements

Curley in Amber (Or, Reason # 54,765 Why AP is DOOMED.)

by matttbastard

Speaking of media dinosaurs, it would appear the Associated Press is once again eager to hasten its own extinction:

I’ve tried to avoid speaking out regarding reports about the Associated Press’ plans for the future. I’ve done so because AP executives and board members have a habit of saying lots of things that are later “corrected” after they stick their fingers in the air and discover the wind is blowing another direction. So I assume everything I hear that’s attributed to “someone at AP” is merely a trial balloon.

However, the article in the New York Times today about AP (or, if you prefer, “the” AP) “cracking down on unpaid use of articles on the web,” attributes the insanity it reports to the CEO of the AP — by name. As he was going on record with the New York Times, I have to assume that he means what he’s saying.

In other words, I feel fairly confident now that it’s okay for me to start calling a nut a nut.

Here’s a quote from the NYTimes.com story:

“Tom Curley, The A.P.’s president and chief executive, said the company’s position was that even minimal use of a news article online required a licensing agreement with the news organization that produced it. In an interview, he specifically cited references that include a headline and a link to an article, a standard practice of search engines like Google, Bing and Yahoo, news aggregators and blogs.”

In other words, what I just did — quote the New York Times and point to the article — would be considered a copyright violation by AP if the point was to an AP story. To quote and link to that story would require me to have a licensing agreement with AP. That policy, of course, is nuts.

And I’m not even saying it’s nuts from a legal “fair use” standpoint or nuts because AP reporters quote and link to bloggers all day everyday. And I won’t even explain why it’s nuts because of the traffic-driving dynamics and economics of advertising revenue that results when I point to an AP story on, say, my hometown newspaper’s website.

I’m just saying “it’s nuts.” And it’s nuts that Tom Curley doesn’t understand why it is nuts.

BTW, remember this little teapot tempest from the stiff-spined TECHNICAL GENIUSES @ AP?UNAUTHORIZED!!!1

Yep — still embeddable.

Golf. Claps.

Methinks some jaunty grave-dancing music is in order:

Recommend this post at Progressive Bloggers

Iran: Dreams Underfoot

by matttbastard

(Image: sterno74, used under a Creative Commons licence)

Following the 2003 invasion and occupation of Iraq, Tom Regan’s Terrorism and Security Briefing for the Christian Science Monitor became a must-read for anyone who wanted a daily general analysis of counterterrorism/counterinsurgency developments around the world. Unfortunately, Regan no longer compiles the briefing. But, late last week, he quietly emerged from an undisclosed location to pen this must-read take on the ongoing post-election turmoil in Iran.

Regan notes that the West may be projecting its own collective desire for transformative political reform in the region onto a murky, still-fluid situation that is not quite the widespread democratic uprising that the mainstream media and Western political establishment would have us believe:

…I strongly believe that what are seeing in Iran is something like a reality based TV show. It’s based on a real incident, but it’s still being shaped by the show’s writers and director (ie, the western media) to be the most interesting to a Western audience. We’re only seeing the bits of tape that conform to what the western media ([which] represent us) want the story to be. It’s real but it’s not reality.

First, this is most definitely NOT a national revolution. This is a protest largely based, as I said, in northern Tehran, the more affluent and prosperous area of the city where most of the universities are located as are (surprised) the hotels where most western journalists stay. As Time’s Joe Klein (who just got back from Tehran) noted in an interview on CNN yesterday, there is no protest at all in southern Tehran, the largest part of the city where the poor and less-educated live. This is Ahmadinejad ’s base. And there is almost no protest at all in rural areas. The regime is firmly in command in most of the country, and the more repressive elements like the Revolutionary Guard have yet to really make their presence felt.

You know, this beginning to sound like Beijing 20 years ago.

Now, there is always the chance that a revolt driven by a relatively small number of the country’s population will succeed in overthrowing the country’s regime. Especially in Iran, where one revolution has already done that. But that was a revolt approved by the large majority of the people against a hated despot. This is not the same situation. If there is hatred of Ahmedinejad it comes no where near close to the hatred felt for the Shah. It’s just not going to happen.

As they say, read the whole damn thing.

h/t Karoli via Twitter

Related: Patrick Martin provides a history lesson on Mir-Houssein Mousavi, a most unlikely champion for Western-style liberal democracy, while John Palfrey, Bruce Etling and Robert Faris of Harvard University’s Berkman Center for Internet & Society share an informative survey of the overall Iranian web presence (which–surprise–may not conform with what we’ve been voyeuristically observing via Twitter). Elsewhere, Dana Goldstein gives us these two mustread posts on the role Iranian feminists have played in the uprising (h/t Ann Friedman). Also see the one and only Antonia Zerbisias (taking a welcome respite from blogging about her thighs and pention [sic]) for more on how–and why–the women of Iran have taken the lead in demonstrations.

Recommend this post at Progressive Bloggers

The Return of ‘Lawful Access’

by matttbastard

Well, isn’t this lovely:

The Conservative government is preparing sweeping new eavesdropping legislation that will force Internet service providers to let police tap exchanges on their systems – but will likely reignite fear that Big Brother will be monitoring the private conversations of Canadians.

The goal of the move, which would require police to obtain court approval, is to close what has been described as digital “safe havens” for criminals, pedophiles and terrorists because current eavesdropping laws were written in a time before text messages, Facebook and voice-over-Internet phone lines.

The change is certain to please the RCMP and other police forces, who have sought it for some time. But it is expected to face resistance from industry players concerned about the cost and civil libertarians who warn the powers will effectively place Canadians under constant surveillance.

Constant surveillance–how so?

The concern of critics is that unlike a traditional wiretap that cannot commence without judicial approval, lawful-access legislation in other countries has forced Internet providers to routinely gather and store the electronic traffic of their clients. Those stored data can then be obtained by police via search warrant.

“That means we’re under surveillance, in some sense, all the time,” said Richard Rosenberg, president of the B.C. Freedom of Information and Privacy Association. “I think that changes the whole nature of how we view innocence in a democratic society.”

Um, yeah, just a li’l bit.

Oh, and, via Michael Geist, it seems our loyal opposition is also doing its part to represent the best interests of the nation by, um, once again proposing its own lawful access legislation–a bill even more odious than the government’s’:

…Liberal MP Marlene Jennings has reintroduced her lawful access private member’s bill, called the Modernization of Investigative Techniques Act.  The Jennings bill is a virtual copy of a failed Liberal lawful access bill that died in 2005.

[…]

[T]he Jennings bill would require ISPs to disclose customer name and address information to law enforcement without court oversight.

The Magical ConservaLiberal Unity Pony drops yet another stinking, steaming load on our heads; I love the smell of bipartisan turdblossoms in the morning.

Cough.  Anyway.

From what I can tell, the only substantive difference between Van Loan’s proposed piece of legislation and the one then-Public Safety Minister Stockboy Day tried to surreptitiously impose in 2007 without any public input (before backpeddling quicker than you can say ‘Ogopogo’) is the apparent requirement of judicial approval (which, as noted, may not provide much in the way of protection for a citizen’s private online information–and  Jennings’ PMB offers, um, none).  Otherwise, the state will, in essence, be forcing ISPs to fulfill the darkest fantasies of the tinfoil-adorned black helicopter set.

And, as Impolitical (h/t) notes:

The dangers of such powers being placed with law enforcement and the potential for abuses have been made abundantly clear by the experience Americans have had with the Bush administration and the revelations from whistleblowers in the last year.

Two examples:

Am in full agreement with Geist here:

…Van Loan should commit to active consultations with the privacy community before introducing the legislation; renew the government’s pledge for full court oversight (including for customer name and address information); and there must be full hearings on the bill that place the burden on law enforcement to demonstrate that there is a problem with the law as it currently stands.

Bottom line: this is not a path any purportedly ‘free’ society should hastily embark upon.

Recommend this post at Progressive Bloggers

Mumbai Attack Resources

by matttbastard

Canadians concerned about relatives/loved ones in Mumbai can call the Department of Foreign Affairs at 1-613-996-8885 from inside Canada or 1-800-387-3124 from other countries. US citizens contact special State Department call center phone number set up for the crisis: 1-888-407-4747. UPDATE For Australians:1300 555 135 or +61 2 6261 3305 (DFAT) UK: 0207 008 0000 (Foreign Office) (h/t Brandy Betz)

Please add additional resources in comments and I’ll update accordingly.

– Indian dead tree media: Hindustan Times, Times of India, The Hindu, Outlook India, Deccan Herald, Indian Express

list of Mumbai bloggers liveblogging events as they unfold, comprehensive round-up from DesiPundit

– Vinu’s Mumbai attacks Flickr photostream UPDATE new sets from Vinu here and here, ashesh shah’s photostream (h/t Gauravonomics) UPDATE Boston Globe photoset (h/t dina)

Google Map of attacks

– regularly updated Wikipedia page

YouTube and VodPod videos, sorted by date (most recent first)

– Ongoing coverage from Neha Vishwanathan of Global Voices Online [UPDATE Global Voices special Mumbai attacks page) and from GroundReport.com

searchable list of injured/dead

– regularly updated emergency information at Mumbai Help

PinStorm information page

– Livestreams from CNN-IBN and NDTV

– Twitter content marked #mumbai, Colaba, Oberoi, Taj

Mumbai Tweetgrid (automatically refreshes)

– invaluable Twitter updates from MumbaiAttacks, zigzackly, vinu, gsik, chhavi, asfaq and dina .

SkyNews MicroBlog

Timeline of terror attacks in India, 1993-present

UPDATE public Google Notebook aggregating key points and facts (thanks, Anannya Deb!)

UPDATE Mahalo , Addictomatic and NowPublic pages

Recommend this post at Progressive Bloggers