Solid

Solid
w3c Solid

Complex Networks

The rising scale of data and the need for information gain has provided a greater need towards understanding patterns to form knowledgeable insights. In many cases, such patterns can be derived through machine learning and data mining. But, also through studying complex networks that form within contextual data. The below links provide useful sources of study in the science of complex networks.

The Structure and Dynamics of Networks

Networks, Crowds, and Markets

Networks: An Introduction

Understanding Social Networks

Six Degrees: The Science of Connected Age

Coursera: Networked Life

Coursera: Social Network Analysis

Network Science

Network Science Book

Brexit Pipeline

Studying Sentiment Analysis in context of Brexit (EU Referendum) is currently an intensive area as the polling stations will very soon be active for voters. Input sources from social media and news feeds can be a focal point for storytelling about the various events. Social media and news feeds can be utilized in form of stream processing which can then be used for machine learning analysis and then indexed for summary into Elasticsearch. A sample workflow example is provided below.  Reader will take notice that the sample workflow is also supported as an example for learning Apache Flink. The workflow can be modified as required for example, one could use a Redis cache layer between the machine learning process and Elasticsearch. Also, could extend with an NLP pipeline (Gate/UIMA) or simply OpenNLP/CoreNLP for extracting information. One could even replace Apache Flink with Spark or GraphLab. Alternatively, one could even replace Kafka with Kinesis and simply apply the AWS data pipeline. Also, the data sources can be stored using S3. Furthermore, one could even use DL4J with Spark on ElasticMapReduce to apply Deep Learning approach in form of convolutional neural network model. Although, Python developers may be more inclined to use Theano, TensorFlow and possibly RabbitMQ. For a graph representation one could use Titan, GraphX, Elasticsearch Graph, Cayley, PowerGraph, Gelly, among others. As one can see there are several ways of implementing a solution on a case-by-case basis to translate the requirements of stories. However, prototype in small is always the best way to go before scaling out incrementally i.e fail fast. 

Input->Kafka->ApacheFlink->Elasticsearch->Output

Steps:

1. Collect
2. Log 
3. Analyze
4. Serve & Store

List of Input Sources:

• Twitter
• Youtube
• Google+
• Facebook
• Tumblr
• IntenseDebate
• Disqus
• Blogger
• YouGov
• Reddit
• NewsFeeds (Reuters, Digg, HuffingtonPost, BBC, Guardian, Telegraph, Independent, Google News, AP, TheWeek, etc)
• EUReferendum
• EU-referendum
• Fullfact
• InFact
• UK & EU
• Breakingviews
• Pollstation
• UK Cabinet Office
• Economist
• Whatukthinks
• Wikipedia
• NCpolitics
• Survation
• ICM
• ORB
• Opinium
• BMG Research
• Ipsos-mori
• UKpollingreport
• MigrationWatch
• European Central Bank
• EconomicsNetwork
• Global Research
• ...and others

As a side note, GNIP and DataSift provide an entire data source pipeline for building out a firehose of streaming inputs. Live Polling data can also be used to gather voting trends as they happen. However, as the referendum is now past, one can probably get a hold of the dataset or API.

Intelligent Recruitment

Recruitment processes at companies need to change. It is a given fact that human resources have no clue about how to recruit candidates when they are not even qualified to know the technical skills required for any given role. Also, recruitment agencies are even worse charging fees for unqualified services where they have literally know clue about employer requirements or the candidate CVs. In fact, it is not even possible that a recruiter would have an understanding of almost all the role types and the skills required for them. The most they can do is count keywords without having a clue as to the context of the skills applied. A candidate has to furnish a CV that looks readable to an unknowledgeable recruiter while also keeping it substantially technical for the interviewer. Scratching at the surface it seems most companies need to do away with the recruiter altogether and use better and more intelligent means at recruiting staff. Secondly, human resources are there to protect the employer not the employee. And, most recruiters only sift through CVs at a glance and possibly not even bothering to check all of the candidate applications. The entire recruitment and human resources function can be automated through a service-oriented or microservices style architecture with an intelligent agent playing the pivotal roles of recruiter and human resource advisor/assistant/manager by using modularized backend functionality. Such measures will not only mean all candidates get an equal chance at employment but it also means a meaningful and seamless recruitment process. Similarly, it also means a seamless human resources experience for employees at an organization. It also means a better experience, a more secure way, and an effective way of finding jobs for candidates with higher chances of them getting their ideal role. In end, employers will not end up reporting on skills shortage where there aren't any especially as there are so many able candidates available in market that can be tapped. If only they used more efficient, intelligent, and responsible means in which they provided a balanced process for all candidates and employees in the mix.

The connected services for an intelligent agent could incorporate:

• CV creation and publishing service
• feedback/survey/complaints service
• compliance and screening service
• candidate/employee/employer review service like glassdoor
• separate application for contingency worker management
• CV database with stringent access controls
• job search functionality that can be queried via SPARQL/Linked Data
• filtering service
• freelance bidding service
• semantic weighted graph matching service that brings candidates/recruiters/employers together
• identity/social/influence profile management for ranking candidates
• microtasking service
• ranking/commentary/news service
• recommendations service
• internal corporate search enablement
• social search enablement
• semantic CV extraction and parsing service
• tracking service like jobvites and workable
• internal human resource management for permanent employees payroll and other services
• additional recruitment and human resources workflow services
• social platform like linkedin or meetups that incorporates aspects of stackoverflow and events
• an ontology for human resources and recruitment to formulate commonsense reasoning
• SKOS thesaurus that can be utilized for term extraction, linking, and annotation
• blockchain for the entire recruitment process
• an ontological understanding of people, cvs, roles, sectors, and skills
• w3c standards compliant process using schema.org and other formats
• an insights and metrics service to understand the distributions in the market and the effectiveness of the hiring process

ICML 2015

This year the International Conference on Machine Learning took place in Lille, France. It was a fantastic event to bring research from a diverse areas of Machine Learning in a collaborative setting. The conference went down really well. There was an immense amount of research shared within the community.  Also, a noticeable increase in number of people that attended the conference this year. The schedule was broken down into conferences, workshops, and tutorials. Even an open question and discussion session was available after each session. The banquet was a joyful experience. However, both the banquet and the local Lille food experience was much to be desired. Cheese was on display, in all forms, and showing itself in every french menu. For vegetarians, Lille offers cheese, french fries, and salad. Some of the most popular areas of research covered included: Deep Learning, Topic Modelling, Structured Prediction, Networks and Graphs, Natural Language Processing, Reinforcement Learning, and Transfer Learning. Deep Learning, Reinforcement Learning, and Word2Vec were the most popular researched topics in attendance. Many of the presented papers can be found also on Arxiv. The conference also showed how far Machine Learning has come as well as the level of popularity it has garnered over the years. Machine Learning is proving to be an invaluable area in a multitude of domains which is having profound effects for business and society as a whole.  But, one thing was reverberated throughout the conference that even now there is still a lot to be discovered before Artificial Intelligence can truly match the abilities of a human being.

NetworkX

Complex networks have become a popular science of big data processing especially for the Web. NetworkX is one such library that provides a flexible option in the python environment to study such graphs in context. The library provides much scope towards large scale and real-world networks especially for social network analysis. The library can be amazingly useful for visualization alongside D3. It is also compatible with Sage and graphs can be integrated.

Semantic Web For Dance

Dancing is deeply ingrained in almost every society on the planet. The art is so varied in routines and styles often providing depth into the cultural diversity of a society. It is an open expression, an exercise in one's freedom, and provides for unlimited ways of displaying creativity as well as a way for humans to identify with their individuality. With the ever growing styles of dancing, it also is an added complexity in keeping up to new changes in moves and routines. It also makes it complicated for learning as well as in keeping track of all the different types. Semantic web could come in handy in area of dance for insurmountable ways. It could help choreographers with planning routines. It could also help enhance learning in dance. It could even help semantically categorize all the different dance styles, trends, and moves as well as provide for better ways to track them. It could also help annotate dance videos and general content on the web for indexing. It could even help in knowledge discovery through linked data. Also, such approaches could go further in historical context towards understanding the cultural diversity, evolution, and the importance of dance in various societies over time. They could even help towards gaining insights for developing even newer dance styles. Dances also have embedded messages within them in form of expression, one can attempt to understand the semantically rich feelings and attitudes of an individual with the moves and choice of routines.

List of dance style categories
List of dances
IDTA
Popular Dance Styles
most popular dance moves from music videos 2013
top 10 list

Online Shopping

The web has become a big hub for ecommerce activity with many contextual products and services on offer. However, there are still limitations of what one can sell online for which many it does become a difficult and an awkward affair. A lot of this is down to our five senses and how difficult it is for us to measure the goods and services in context. At other times, such goods and services may just be classified as illegal for which there is an obvious regulatory block. Consumers generally want to be able to hold, smell, taste, and see products before they buy. How does one measure the comfort of a mattress online? How does one check the expiry of dairy products before buying online? How does one check the freshness of meat, vegetables, and fruit online? How does one go about buying lipstick online? How does one know that a pair of shoes fit comfortably? How does one know that the pair of slacks or shirt actually fits well? Even the likelihood of ordering a tailored suit online are quite slim. However, there will always be people that are just plain lazy, have no time to spend on grocery shopping, or going out to try on clothes before buying. People obviously want to be able to experience online shopping like they are physically there but in comfort of their home. More and more people are taking advantage of the flexibility as they shop online at their leisure. Ecommerce has become a big business but with it also brings certain levels of awareness of meeting customer expectations as well as complexities in window dressing while drawing a virtual mirror into the five senses of the human perception. Also, the constant need to stay competitive is another hurdle. Often times prices online are set much cheaper than in store until there is an obvious sale. Online shoppers are also generally a lot smarter with their buying habits, at least in some respects. Customers also have the luxury of using third-party applications that can provide assisted tools for shopping such as in aggregation of online bargains. There is always a constant need and necessity of ecommerce businesses to stay competitive and in order to stay competitive there is a need to monitor the market and consumer behavior. Ecommerce as a result makes a huge return from big data analytics both in form of social media, advertising, as well as general customer engagement efforts. Businesses online seek to entice customers and increase online traffic to their sites while customers want to find the right product or service that meets their needs. The Internet provides businesses an instant global platform to showcase their products and services with a global consumer base. There are also regional limitations and restrictions to conform to in respect of trade regulations. There is also complexities in consumer behavior from region to region which influences the online store front. Ecommerce also faces many complexities in security and the way they manage consumer transactions. All in all, as billions of transactions are conducted online in all array of contexts, there is a need to analyze consumer behaviors in rich information flows, but at same time providing a connected and satisfying experience for all. As knowledge discovery grows leaps and bounds through artificial intelligence, we are likely to see more breakthroughs in technological advancements to emerge, to influence, and add further dynamics to the ecommerce domain. 

Quora

Quora is a site where questions and answers are collaboratively discussed and modified through a community of users. Essentially a social process. However, this social process is flawed and at times opinions are taken personally from users which means the site lacks integrity in respecting every ones' opinions and not of specific few. In that process, it discriminates on opinions for answers to questions. Which in a lot ways diminishings the whole collaborative and social aspect of the process. The model is flawed and the answers will often not be objective. It also means people will get into heated discussions when they take things too personally which can be derived as anti-social behavior by personal attacks on individuals. The moderation process is also very discriminating and perhaps can also be viewed as bigoted as their opinions on evaluation are also surmised in discriminating users. It seems in all intents and purposes if one cannot handle a variety of opinions then perhaps it is not the best place to be as one will often end up taking things too personally. There is a rather big social issue on the internet that arises from people taking the aspect of comments seriously losing the perspective and context of reality. It may even be taken as the fact that an opinion is every ones essential right whether it is agreeable or not. However, what turns into anti-social behavior is when personal attacks are taken on which is not very conducive to the openness of the social web on the basis of which the internet has been so successful. Perhaps, the view of social media and networks needs to be taken with a more open view by users when using such sites knowing that there will often be views that they may not agree with but having the sense of respect to understand that others have a right to hold their own opinions whether right or wrong. This is one failing factor of social web where people blur the lines between reality and internet and often end up taking things too personally. It is also perhaps why there needs to be a certain awareness of such issues as well as understanding that the global space like the internet necessarily will have variety of content not necessarily agreeable to all. Such sites also display a very significant level of bigotry in the way they define terms of service through discrimination and the way content and users are moderated. It often leaves one with a bad taste when such sites blur the perspectives and lose the whole defining objective of an open social web. Maybe, one way to avoid such sites is to not support their use until they are able to provide an equal and open approach for the way they handle users and content. In general, the collaborative approaches are not effective towards objectivity. They are also not efficient for establishing constructive and accurate ways towards intelligent solutions. For open question and answering as well as recommendations the approaches need to utilize semi-automated techniques to provide for better alternatives. They even could use a pure automated approach but that avoid social collaboration entirely. In a manner of speaking, social collaboration is only as good as when it enriches or adds value to an algorithm as part of human assistance, but they should be necessary in guiding an intelligent system towards a logical conclusion. Quora as a social platform does not work nor does it provide accurate answers. But, what it does do is provide variety of stagnating inputs in form of collaborative insightful answers which can be classed as opinions with very little added intelligence - not a very smart solution to an uncomplicated problem. Alternatively, they could provide an open question/answer search function alongside the community of answers that would add richer contextual value towards building a more natural semantics as well as discoverable analytics. An ontology could increase in harnessing more semantics as well. The natural step, in process, would be to spontaneously link the social web of discussions via linked data through which an evolving graph emerges.  

Dating Sites

It is a sad world where people find it difficult to even meet in person that they have to go on dating sites. There are dating sites and match makers all over the internet. Each providing some form of stickiness. However, they all want the person to subscribe in some way or another. It is even worse when someone has to pay a subscription to meet people online. It is also not the safest option. Internet is one place where free speech and opportunities abound. However, there are also people out to exploit others. There has to be a better approach to people connecting with like minded individuals. Meetup is perhaps an even better approach. At least, the process gets people connected at a very contextually specific level and transforms it into a personable group activity. But, it is still in essence a very disconnected social experience. Perhaps, a better alternative might be to use semantic web with all its metadata and build a linked data of communities. Similar to linked data of discussions. The social web has endless possibilities. People can even build their connections via mobile phone and even connect in real-time using bluetooth while they are out and about. One will never know another by just meeting them online. It has to be an in person encounter. Dating sites have never really been all that popular either on the web compared to social networking sites like facebook and twitter. There is also a real need here towards a more semantically connected identity allowing people to be more mobile but at same time safe from fraud. Dating sites can also be a very shallow place to meet people in similar respects to facebook. Perhaps, this can be equated to a major drawback of capitalist society. People tend to become more individual in their mind set. But, at same time there are many that end up living a very separate existence. This is typical of big city living where people may appear cold towards others and more vigilant of their surroundings. Dating sites also appear to be more popular by people living in rural areas rather than urban. This is in many ways an indication of how people living in different parts of the world feel drawn and connected to the opposite lifestyles out of curiosity. People also live busier lives compared to the way people lived and worked decades ago. But, there has always been a need in people to share their existence with someone else. Surely, dating sites cannot be the answer if bars and clubs are so popular in a city. Perhaps, lack of services, busy family lives, and boredom in the local communities draws individuals on to such sites.  But, building another disconnected site is not the answer any more to a social web. 

Linked Data for Discussions

Disqus is an interesting platform for the web communities of discussions. They can be used in fundamentally three different ways: using the Disqus API as a point of integration, via a JavaScript code, or direct integration into the supported blog platforms. It is a nice enough feature getting traffic to a site as well as to collaborate on discussion flows. However, this does not lend itself to all. Disqus requires you to host your comments as a third-party which might be risky for some as they lose control over when as well as how their comments data will be used and made accessible. Also, it has the potential of slowing down a site considerably.  One can also lose their comments once removed from the site. It also relies on JavaScript quite extensively which can add to much security risks. An alternative option is to use IntenseDebate which has a nice integration to Blogger and even Wordpress. However, a lot of these discussion platforms are diverting data away from the hands of the user in a centralized way and making it less connected. What would be a better option is towards a decentralized control of comments and discussions in a distributed fashion towards a linked data approach. That way each site controls its own data on comments as to what to expose and what not to. But, also it allows for comments to be linked with other sites in a web of data. Perhaps, even utilizing an extensible generic comments API. One can then query for comments and find open answers and even develop sentiment analysis. Connecting linked data as a discussion forum means a vast social network of comments, that are annotated, which can be harnessed, and that allow people to globally connect based on interests, influence, and other contexts. It also means people maintain a sense of security and control of their own data. Information is a valuable commodity. But, information is even more valuable when it is contextually enriched with other data sources semantically. Even social networks should really be interconnected. We do have such options as OAuth which allow for socially connected authentication. Even such options as linked data profile. Web of communities essentially means linked data of communities and that essentially means a social web of data at the hands of anyone that wants to query. The load of that query can then be distributed via the decentralized sources and control of data. This also means websites can be even more search engine optimized and accessible for social content. It also means a better semantically available social graph for navigation and analysis to all without losing or compromising on security at the hands of a few.

Other Alternatives:
LiveFyre
Echo
Facebook Comment Box
Google Plus Comments
Realtidbits
Comments Plus
Moot
CommentLuv
TalkaTV
Juvia
Burnzone
SO: Unobtrusive Self-Hosted Comments
Tidlehash
Isso
and more...