28 April 2016

Data Science of Colors

Colors play an important role in data science in clarifying and visualizing of an information gain for a data-driven story. They magnify and project insights from data adding much needed value. The following are some highlighted links on color palettes.

kuler
colorlovers
colorbrewer2

27 April 2016

Public Data Sources for Machine Learning

UCICollection of benchmark datasets for regression and classification tasksUCI Machine Learning Repository
KDDExtended version of UCI datasetsUCI KDD Extended Version
DELVEPlatform for comparative assessment of regression and classification tasksDELVE
DMOZCollection of links for different datasetsDMOZ Directory
KDNuggetscollection of links for different datasetsFurther Datasets
ChemDBchemical data that can be used as datasets for machine learningChemDB
Golemtrying to learn rules for predictionGolem Datasets
NDRData sets for nonlinear dimensionality reductionNonlinear Dimensionality Reduction
GeneralA list of dataset links by categoryfurther datasets
AWS Publicpublic list of datasets via S3large dataset repository
Datahubpublic list of datasetsdatahub datasets
BigMLcurated list of datasetsbigML datasets
Curated Githubcurated categorized list of datasets on githubpublic datasets on github
wikipedia listcurated categorized list of datasets on wikipediadatasets of ML
Data ScienceData Science Projects19 free public data sources
Data ScienceData Science Projectsdata science datasets

31 March 2016

JavaScript Ecosystem

JavaScript ecosystem is huge and it gets even bigger when one includes Nodejs applications. Keeping track of new trends can be difficult as it grows in so many different directions at such a rapid pace. One can obviously keep abreast of the changes through community meetups and even Github for that matter. In fact, choosing the right library for an application can pose a dilemma as one is just so spoilt for choice when it comes to JavaScript. However, there is no official standardization in place apart from the actively 'worked on' ECMAScript. It is like a mushroom cloud of libraries in the community that just keeps getting bigger. Also, as new libraries evolve, others seem to die out or lose traction with complete lack of support. There seems to be no formal quality assurance or a standards driven approval process like some other languages. Apparently, it seems JavaScript community is very much driven by trends and at times that can even dictate the choice of libraries used in the applications as a form of value or impact to the business. Naturally this also adds a degree of risk. The current trends are of reactive applications. The following links provide some aggregated view of Javascript ecosystem and the different trends for the various libraries.

Javascripting
Libscore
Libscore Search
List of JavaScript libraries
awesome javascript
whats happening in the javascript ecosystem
what to expect from javascript in 2016 beyond the browser

27 March 2016

Intelligent Publishing

Many aspiring writers get rejected by publishers. They pick themselves up and try again after having submitted drafts to multiple small and large publishers. Book publishing is an art form just like any other. One needs to be able to craft a perfectly imperfect story. However, editors are imperfect too. There should be more opportunities for writers to convert their thoughts and monetize them without anyone trying to block their talents. Books also go through digital standards like Onix for publishing on demand. There needs to be more intelligent solutions in place that brings authors, publishers, and readers together. Readers also demand certain stories as trends and tastes change over time. A certain degree of sentiments also are held by readers in respect of how stories evolve. Self-publishing has also evolved into providing more freedom for authors. Ebooks is providing for new and bigger challenges for publishers. There are many ways in which intelligent agents, and for that matter, artificial intelligence could help in this digital age of publishing especially as the cloud garners such momentum. Stories take on many forms from news articles to books to even internet. Firstly, social media could be utilized to bring storytellers together to collaborate on the development of stories. Furthermore, there is even opportunity here for collaboratively building scripts for a movie or drama serial. An intelligent agent in this context could guide their storytelling ability in the right direction in such ways as adjusting plot lines, adding better starting/ending to a story, organization of chapters, and even the development of characters. The intelligent agent here could act as a live editor often providing ideas and guidance in the process. Once the joint collaborative effort is complete it could be published as an ebook or an escript and earnings shared between the contributors. However, it is not necessary to collaboratively build a story and many authors still want the level of total control. There could even be a marketplace for authors and publishers. One could use the web to create awareness of the book and even gather reviews during the process as well as after publication. One could even use the social media to learn the types of stories people really want to read. Authors can also pick on ideas and sentiments of what readers feel about their book or article. And, intelligent agents could be taken further as to develop pattern recognition on story formulas for a bestseller. Many people maintain diaries which could further add to the process of assimilation of thoughts and ideas for a story. In fact, the diary could take on the added role of an intelligent personal assistant that could provide additional services of organizing the day, thoughts, even at times of a writer's block. There is also a need here for a flexible reader/writer template that could add additional functionality for authors/readers in form of annotations, entity extraction, semantic understanding, thought bubbles, flexible notes, mind mapping, linking, drag and drop, grammar checking, spelling, machine translation, fact checking, dictation and other options. Reading and writing stories should be fun. Crowdsourcing for content is another option which are utilized on sites like Yahoo. However, such content needs to be quality assured, fact checked, and objective. Journalists should be able to sell stories on the open marketplace to news agencies. Intelligent agents could be used here as well to assist both journalists and readers. For readers, an intelligent agent could provide for an optimized summary of news articles, timeline of events, recommendations, and even personalization service for content. For journalists, an intelligent agent could provide recommendations on breaking news, fact checking, story tracking, optimization of headlines, and structure to a piece of story. Content Advertising is also a very big factor here in reducing costs and providing further incentives. Back office functions already utilize more service-oriented and rules-driven on demand publishing systems which could be enhanced using linked data patterns. Involving everyone in the process from readers, writers, to publishers makes the publishing fun, more accessible, and a viable endeavor for all parties. Publishing is a fast, competitive, and challenging industry. There is much in the way of standardizations and progress in the area especially inspired from the perspective of intelligent agents which could be incorporated in the entire workflow and provide for more creative options as well as a source of enrichment to the publishing community. 

25 March 2016

Data Visualizations

Making sense of raw data is important in data science. How data is presented can elaborate on a story. Storytelling has become an art form in translating and conveying the meaning of data which is derived as information. In many respects, JavaScript has become an indispensable approach to extensible visualization from data. In fact, data visualization also provides for an interesting approach to understanding the context of data even from view of network science. There are many tools available in community both open source to commercial, and some even tailor for non-developers as well as developers. The sheer richness of libraries and tools in Javascript can make one spoiled for choice. In end, it boils down to the right tool for the job. Although, one can also use Python's NetworkX, GnuPlot, Matplotlib, GGPlot, and GGPlot2 from R to decipher the same data. The below is a list of a few JavaScript libraries, and some selection of further links to design tools in review.

D3.js
NVD3
Dygraphs
Gephi
keylines
Vega
Processing
InfoVis
Google Visualization API
Springy.js
Polymaps.js
Dimple.js
Sigma.js
Raphael.js
gRaphael.js
Leaflet.js
Ember Charts
Visit
Cytoscape
jsnetworkx

selection data visualization
showcases data visualization
design tools data-visualization
the 14 best data visualization tools

24 March 2016

App Stores

There are many app stores on the web now. The three primary ones include Google Play Store, Mac App Store, and Amazon App Store. These seem to have the largest market share and potentially ones with good monetization potential. However, it seems app stores still need to converge, integrate and provide better services. So, far only Mac App store has a decent enough quality assurance process in place. And, Amazon App store provides the widest amount of device coverage as long as it is Android specific. There are certain functionalities that all app stores need to provide these days in order to make life easier for developers.

  • An app store that can support many devices, languages, and platforms not exclusive to a few
  • A separate analytics option to measure app performance and churn rate
  • A separate analytics option to measure sentiments, marketing, and monetization channels
  • An app integration option to drive packaging with ease across multiple platforms
  • A demo section where consumers can get a preview of the app before buying or downloading
  • A docking facility to support on browser, desktop, and mobile
  • A gateway or fusion service that seamlessly makes upgrading, deployments, and publishing easier
  • A widget and theming service
  • A push notification, synchronization, and upgrade management service
  • A separate option for desktop and browser
  • A separate option for mobile, tv, and other ubiquitous devices
  • A better way of monetization for free apps and a way to control viral amplification on the web
  • A secure way to download and upload apps, and app security checking during QA review process, as well as continuous identity and apps screening
  • Also, a standardization process across all app stores to make monetization and publishing easier
  • A more secure and flexible payment processing workflow both for seller and buyer
  • A better semantically searchable option for apps that can be queried across app stores
  • Access to public and private controls for downloads for enterprises, DRM, and country restrictions
  • Age and parental control restrictions
  • Ability to check app compatibility on the fly
  • Simpler and more accessible policy and code of conduct guidelines for developers
  • Better selection criteria for app stores
  • A central administration console
  • Device registration and management
  • Auto-scalability, CDN, and uptime features via public and private cloud options
  • Options for added promotions, offers, and discounts for further upselling on the apps 
  • Also, app stores should only be commission based no flat fee for membership should be necessary, and if the fee is necessary then it should be fairly nominal like $9.99 rather than $99 per year

22 March 2016

Intelligent Agent for ECommerce and Retail

Another intelligent agent use is in ecommerce and retail. The following are a few business cases and service roles that an intelligent agent could provide that can be very useful and in some cases are already actively in use. In some cases, virtual shopping would be ideal. For example, virtually try out clothes or see how a sofa fits in the living room. There is a degree of augmented reality that could add value to the shopping experience. Perhaps, an added distraction of fun while shopping. 

Retail and ECommerce:
  • In-store multi-agent security
  • In-store profiling for individualized customer experience and promotions 
  • In-store shelving and window dressing
  • In-store sentiment monitoring for customer service and experience
  • Out-store sensing of potential conversion of customers
  • In-store sensing of potential curiosity conversion of customers
  • In-store purchases with EReceipts with loyalty promotions, offers, and discounts
  • Personal Assistant
  • Warehouse Agent
  • Delivery Agent
  • Cashier/Check-Out Agent
  • Health, Wellness, and Nutrition Assistant
  • Product Recommendations (Personalization, MorelikeThis, Deep Insights, Social Insights)
  • Semantic Categorization for Search Relevance
  • Stock and Inventory Monitoring
  • Order Fulfillment
  • Loyalty EWallet Service for Fast Transactions
  • Price Monitoring (Price Recommendations, In-Store/E-Store Bargain Analysis, Special Offers)
  • Real-Time Insights (On-shelf, Brand/Product, Promotions, Competitors, Shopper Behavior)
  • Virtual shopping and product tabs for home use