Dissertation pivot

Dissertation pivot

Dissertation, Research
To align my dissertation efforts with the strategic and tactical needs of my employer, Spektron Systems - who is incredibly supportive, I must pivot my efforts.  It is fortunate that I am working with a company that directly uses machine learning and has readily available problems addressed by my research.  I should only have to conduct a small pivot that narrows my research to something relevant. Narrowing my research The direction of my research has been the relationship between the ideas of computational creativity and transfer learning.  In particular, I was looking at transfer learning as the mechanism for computational creativity.  This is a vast problem and unlikely to be useful in the short-term for Spektron. Computational creativity, as a concept, may fit the strategic activities of the company, i.e.,…
Read More

Transfer Learning for SDAR models from small datasets

Dissertation, Machine Learning, Research
Developing representational and predictive models for SDAR (structural descriptor - activity relationships) on small datasets is a problem for in-silico modeling of compound efficacies in drug discovery and design. While there are large sets of toxicity data available, the information about the effect of a compound when related to a human activity endpoint (e.g. reduction of symptoms) comes from clinical trials data and reports in the market. The relative number of data points for efficacy is low compared to toxicity due, in part, to the relatively small number of drugs making it to market. The limited number of examples makes it difficult to train robust machine learning models especially with techniques that traditionally require many observations. Using such techniques; however, is desirable because of potential non-linearities in the relationships. Therefore,…
Read More

Dissertation update – October 2015

Dissertation, Updates
Quick review: I have a working version of a relational auto-encoder and have used it to learn a transfer function between two reinforcement learning tasks.  As has been done in other research, I've used state-action-state triplets as the training data.  My hypothesis is that a relational auto-encoder will build a common feature space for the transition dynamics between the two different reinforcement learning domains. There are two problems that I'm trying to solve.  Both deal with the learning algorithm for the interdomain mapping function.   The first addresses how the data enters training and the second is in the characteristics of the data once run through the trained model. Dealing with uncorrelated data Currently, the relational auto-encoder learns on pairs of triplets presented together by using standard back-propagation.  This approach could have…
Read More

Analogy is a core concept in cognition

Dissertation, notes, Research
We begin with a couple of simple queries about familiar phenomena: “Why do babies not remember events that happen to them?” and “Why does each new year seem to pass faster than the one before?” I wouldn’t swear that I have the final answer to either one of these queries, but I do have a hunch, and I will here speculate on the basis of that hunch. And thus: the answer to both is basically the same, I would argue, and it has to do with the relentless, lifelong process of chunking — taking “small” concepts and putting them together into bigger and bigger ones, thus recursively building up a giant repertoire of concepts in the mind. How, then, might chunking provide the clue to these riddles? Well, babies’ concepts…
Read More
Notes from Ammar/Mocanu – Automatically Mapped Transfer Between Reinforcement Learning Tasks via Three-Way Restricted Boltzmann Machines

Notes from Ammar/Mocanu – Automatically Mapped Transfer Between Reinforcement Learning Tasks via Three-Way Restricted Boltzmann Machines

Annotations
Citation H. Ammar and D. Mocanu, “Automatically Mapped Transfer Between Reinforcement Learning Tasks via Three-Way Restricted Boltzmann Machines,” Mach. Learn. …, 2013. Abstract Existing reinforcement learning approaches are often hampered by learning tabula rasa.  Transfer for reinforcement learning tackles this problem by enabling the reuse of previously learned results, but may require an inter-task mapping to encode how the previously learned task and the new task are related.  This paper presents an autonomous framework for learning inter-task mappings based on an adaptation of restricted Boltzmann machines.  Both a full model and a computationally efficient factored model are introduced and shown to be effective in multiple transfer learning scenarios. Quotes & Notes Re:Random or not Unfortunately, learning in this model cannot be done with normal CD. The main reason is that if…
Read More
Random or Not?

Random or Not?

Updates
One of the basic questions that needs to be answered about using the autoencoder architecture to learn a mapping function between two domains is a question of randomness and of what model the autoencoder is learning. Do I have to pair correlated SARS samples together for input or can I, as with a probabilistic model  (See Ammar for TrRBM.), introduce pairs randomly?  
Read More
Training a gated autoencoder

Training a gated autoencoder

Updates
I trained a gated auto-encoder using code mentioned in "Gradient-based learning of higher-order image features" (See: http://www.cs.toronto.edu/~rfm/code/rae/index.html) There are two input layers.  I used 5000 random quadruples <s,a,s',r> from a MountainCar Task and 5000 from an Inverted Pendulum.  Discrete actions were converted into separate features per action through binarization.  Samples were paired randomly by sorting.  All input features were normalized. The training results are shown in the cost vs epoch graph here.  I want to check these results next.
Read More

Preparing data for training an autoencoder

Dissertation, Research, Updates
I'm training a relational autoencoder using data from two Markov Decision processes tasks: the Mountain Car task and the Inverted Pendulum (or Cart-Pole) task.  To do this, I need to map the sample data to nodes in the input layer (and output layer) in the relational autoencoder.  I see this as follows: Mountain Car Position - real/real Velocity - real/real forward action - boolean {0,1} / sigmoidal backward action - boolean {0,1} / sigmoidal coast action - boolean {0,1} / sigmoidal Cart Pole  θ - real/real ω - real/real x - real/real v - real/real force action - real/real My chair and I discussed the actions as being discrete, but most implementations have a variable force as an action.  So, we're going to address it as such.  This gives a different action space than…
Read More
Notes from Memisevic – Learning to Relate Images

Notes from Memisevic – Learning to Relate Images

Annotations, Dissertation, General, Research
Citation Memisevic, Roland. “Learning to Relate Images.” IEEE Transactions on Pattern Analysis and Machine Intelligence 35, no. 8 (2013): 1829–1846. doi:10.1109/TPAMI.2013.53. Abstract A fundamental operation in many vision tasks, including motion understanding, stereopsis, visual odometry, or invariant recognition, is establishing correspondences between images or between images and data from other modalities. Recently, there has been increasing interest in learning to infer correspondences from data using relational, spatiotemporal, and bilinear variants of deep learning methods. These methods use multiplicative interactions between pixels or between features to represent correlation patterns across multiple images. In this paper, we review the recent work on relational feature learning, and we provide an analysis of the role that multiplicative interactions play in learning to encode relations. We also discuss how square-pooling and complex cell models can…
Read More
Notes from Memisevic – Gradient-based learning of higher-order image features

Notes from Memisevic – Gradient-based learning of higher-order image features

Annotations, Dissertation, Research
Citation Memisevic, Roland. “Gradient-Based Learning of Higher-Order Image Features.” Proceedings of the IEEE International Conference on Computer Vision (November 2011): 1591–1598. doi:10.1109/ICCV.2011.6126419. Abstract Recent work on unsupervised feature learning has shown that learning on polynomial expansions of input patches, such as on pair-wise products of pixel intensities, can improve the performance of feature learners and extend their applicability to spatio-temporal problems, such as human action recognition or learning of image transformations.  Learning of such higher order features, however, has been much more difficult than standard dictionary learning, because of the high dimensionality and because standard learning criteria are not applicable.  here, we show how one can cast the problem of learning higher-order features as the problem of learning a parametric family of manifolds.  This allows us to apply a variant…
Read More