RE WORK - Boston Deep Learning Summit

I have recently attended the Deep Learning Summit in Boston. The event was organized by RE WORK. RE WORK was founded in London. The team is all women. The mission of the RE WORK team is to encourage conversations around entrepreneurship, technology, and science to shape the future. 

This is a quick recount of the event from my perspective.
First of all, I have never been to Boston. The public transportation that I took from the airport to the place of the conference was really easy to navigate (In short, I did not get lost). This is probably a result of the effort put in by the local government to make Boston a premier conference venue. Traffic congestion is another story.

Schedule of Talks
The conference schedule is packed.  The speakers are researchers from some of the top tech companies. Facebook, Google, Amazon, Ebay, and Spotify are all represented. I was excited about two topics in the schedule. Here are some of the papers presented. The papers I chose below are some of the research which I thought was significant to some of the efforts we are doing for our business partners:

      The end product of this research is the MovieQA dataset which contains several sources of information  – video clips, subtitles, scripts, plots, and DVS. One of the representations that were used to build the MovieQA data set is SkipThoughts. SkipThoughts uses a recurrent neural network to capture the underlying sentence semantics. The SkipThought model [2] was inspired by word vector learning using the skip-gram model. Instead of just using words to predicting the context around it, SkipThought encodes sentences to predict sentences around it. The model is based on an encoder-decoder architecture. The encoder maps the English sentence into a vector. The decoder then conditions on the encoded vector to generate a translation for the source English sentence.

Domain Separation Networks lets you adapt your model to a target dataset which you did not train on. Domain adaptation's goal is to train on a "source" dataset then apply the model to a "target" data set generated from a different distribution [3]. The motivations behind domain adaptation are the following: 1. Create high-quality datasets for training is costly and time-consuming. 2. Certain annotation tasks are impossible to scale. Domain adaptation allows you to train on synthetic data then apply the model to the real domain.

References
[1] Distributed representations of words and phrases and their Compositionality, https://arxiv.org/abs/1310.4546
[2] Skip-Thought Vectors, http://arxiv.org/abs/1506.06726
[3] Domain Separation Networks, https://arxiv.org/abs/1608.06019 


Location: 606 Congress St, Boston, MA 02210, USA

Comments

Post a Comment

Popular posts from this blog

OAuth 1.0a Request Signing and Verification - HMAC-SHA1 - HMAC-SHA256

Image
This is a security project that I did this Fall semester. I was interested in why the OAuth1.0a specification did not require you to sign a request using RSA-SHA1 or any RSA algorithm. The first issue that I found is where would you store your private keys in a javascript client? - as a javascript encrypted file ? from the browser ?  - that will be a security hole if you allow javascript to get private keys. These are some of the questions that got me asking and which I cannot find answers to for now. OAuth 1.0a Signature Signing and Verification using JavaScript Implementation Background The OAuth specification tries to define an API authentication method that allows consumer websites or applications to access protected API Resources from a Service provider. The advantage that OAuth provides is that the user of the website or application is not required to disclose their credentials to their service provider. Another advantage of using OAuth to authenticate is that it does not
Read more

Spark DataFrame - Array[ByteBuffer] - IllegalAurmentException

IllegalArgumentException - ByteBuffer - Spark DataFrame I was processing a several million documents (~ 20 million) in which we need to extract the NLP features using NLP4J, OpenNLP, and WordNet. The combination of the three NL features blows up each record to 11 times its original size. We are using all three because we do not know yet what feature sets will be helpful to us. The original dataset is in parquet files in HDFS (16 partitions). I thought that was convenient just use withColumn and pass a UDF (User Defined Function) on the column where it needs those features. withColumn adds the calculated column back to the DataFrame. So I created the spark job (I am on Spark 1.5.2-cdh5.5.2)for the above, and things started to get nasty. I am blowing up the ByteBuffer array on the in-memory columnar storage. This is the exception that I am getting. There seems to be no reference in my code in this stack trace. java.lang.IllegalArgumentException at java.nio.ByteB
Read more

Gensim Doc2Vec on Spark - a quest to get the right Vector

Ever since I joined the R & D group we have been doing a lot of cool things, like trying IBM Watson (see previous blog entry). Now we are doing a lot of Natural language processing.  We wanted to compare the similarity of two documents. There is this excellent project Gensim ( doc2vec ) that easily allow to you to translate large blocks of text to a fixed length feature vector to make comparisons. Here is the link to the  original paper  from some people from Google that explains the approach. In essence, they wanted to find a representation that will overcome the weaknesses of the bag of words model. The doc2vec approach proves to be a reliable approach to comparing similarities of documents because it takes into consideration the semantics and the order of the word in context. So with that, we wanted to use it for a corpus of 26 million documents. Calculating the doc2vec for 26 million documents is not a small task, so we need to process it in Spark. The problem is that there
Read more