7 October 2018

Types of Deep Learning

TypeGroup
Attentional InterfaceAttention-Memory
Memory-Attention NetworksAttention-Memory
One-Shot Associative MemoryAttention-Memory
KeyValue Memory NetworksAttention-Memory
Compositional Attention NetworkAttention-Memory
Deep Memory NetworkAttention-Memory
Structured Attention NetworkAttention-Memory
Hyperbolic Attention NetworkAttention-Memory
Multi-Cast Attention NetworkAttention-Memory
Bi-Directional Attention FlowAttention-Memory
Variational AutoencoderAutoencoder
AutoencoderAutoencoder
Denoising AutoencoderAutoencoder
Sparse AutoencoderAutoencoder
Contrastive AutoencoderAutoencoder
FeedforwardBasic
PerceptronBasic
Multilayer PerceptronBasic
Deep Convolutional NetworkCNN
Convolutional Deep Belief NetworkCNN
Convolutional GANCNN
DeConvolutional NetworkCNN
Deep Convolutional Inverse Graphics NetworkCNN
Geometric Deep LearningCNN
Convolutional Kernel NetworksCNN
Convolutional AutoencoderCNN
Hierarchical Convolutional Deep Maxout NetworkCNN
Deep Belief NetworkDBN
Continuous DQNDQN
Deep Q NetworkDQN
Dueling DQNDQN
Episodic-Memory DQNDQN
Bidirectional LSTMLSTM
Convolutional LSTMLSTM
Grid LSTMLSTM
Long Short Term MemoryLSTM
Peephole LSTMLSTM
Phrasal LSTMLSTM
Hierarchical LSTMLSTM
Gated Recurrent UnitLSTM
Adaptive Resonance TheoryModular
Maximum EntropyModular
CounterpropogationModular
SplineModular
GaussianModular
NeocognitronNeural
Neural ProgrammerNeural
Neural Turing MachineNeural
Neuro-FuzzyNeural
NeuroevolutionNeural
Neural Associative MemoryNeural
Neural Hawkes Process MemoryNeural
Sequence-2-SequenceOther
Deep FeedforwardOther
Deep Neural NetworkOther
Helmholtz MachineOther
Hopfield NetworkOther
Kohonen NetworkOther
Compound Hierarchical Deep ModelOther
Dense Associative MemoryOther
Hierarchical Temporal MemoryOther
Large Memory Storage and Retrieval NetworkOther
Generative Adversarial NetworkOther
Associative Neural NetworkOther
Adaptive Computation TimeOther
Deep Coding NetworkOther
Deep Deterministic Policy GradientOther
Deep Predictive Coding NetworkOther
Deep Reservoir ComputingOther
Deep Residual NetworkOther
Deep Stacking NetworkOther
Diffusion NetworkOther
Echo state NetworkOther
Elman Jordan NetworkOther
Extreme Learning MachineOther
Instantaneously Trained Neural NetworkOther
Learning Vector QuantizationOther
Liquid State MachinesOther
Spiking Neural NetworkOther
Tensor Deep Stacking NetworkOther
Radial Basis FunctionOther
Recursive Neural NetworkOther
Markov ChainProbabilistic
Deep Bayesian Neural NetworkProbabilistic
Deep Markov ModelProbabilistic
Stochastic Neural NetworkProbabilistic
Spike and Slab RBMRBM
Boltzmann MachineRBM
Restricted Bolzmann MachineRBM
Bidirectional RNNRNN
Clockwork RNNRNN
Continuous Time RNNRNN
Dilated RNNRNN
Hierarchical RNNRNN
Recurrent Neural NetworkRNN
Second Order RNNRNN
Multi-Time Scales RNNRNN
Recurrent Multilayer PerceptronRNN
Deep Kernel MachineSVM
Support Vector MachineSVM
Shallow Neural NetworksThoughtVectors/WordVectors

*Shallow = one hidden layer in NN
*Deep = more than one hidden layer in NN

7 July 2018

Mailer Campaign Uplift Modeling

Profit(C) = ExpectedProfit(C) x [P(B | V) - P(B | C)] - AdCost(C)

  • P(B | C) - probability of buying given control without ad campaign (Naive Bayes)
  • ExpectedProfit(C) - profit to make from customer if they decide to buy (Regression)
  • P(B | V) - probability of buying given variant of ad campaign (Naive Bayes)
  • AdCost(C) - cost to mail campaign to customer as a constant
  • likely to take into account market or customer segmentation
  • regression could be either logistic or linear
  • total profit would be determined by how much the customer decided to buy either with control and/or ad campaign
  • optimization of ad campaign given the customer conversion ratio
  • use customer data as part of expected profit measures for average spend
  • additionally, more ways to approach the same contextual measures of profit

3 July 2018

Test-Driven Machine Learning

TDD -> Kent Beck
BDD -> Dan North
Refactoring -> Martin Fowler
Agile -> James Shore

Random processes in machine learning need to be measured and controlled, various simple testing strategies can make this possible.