Mabble Rabble: sentiment analysis

Showing posts with label sentiment analysis. Show all posts

22 March 2025

Text-Driven Forecasting Papers

Business Decisions with Sentiment Analysis

3 March 2025

Applied Text-Driven Forecasting

In an era defined by the relentless deluge of data, traditional forecasting methods, reliant primarily on numerical time series, are increasingly challenged. The digital age has birthed a vast, largely untapped resource: unstructured textual data. From social media feeds and news articles to customer reviews and financial reports, text holds a wealth of information that can significantly enhance predictive accuracy. Text-driven forecasting, therefore, emerges as a powerful tool, capable of extracting valuable insights from the narrative fabric of our world.

The core principle behind this approach lies in the recognition that human sentiment and expressed opinions are potent precursors to future events. For instance, a surge in negative social media commentary surrounding a product launch can foreshadow declining sales, long before traditional sales figures reflect the trend. Similarly, a noticeable increase in positive news coverage about a specific industry might indicate impending growth and investment opportunities.

The methodology employed in text-driven forecasting typically involves several stages. First, large volumes of relevant text data are collected and pre-processed. This stage includes tasks such as cleaning the text, removing irrelevant information, and standardizing the format. Next, natural language processing (NLP) techniques are applied to extract meaningful features from the text. This can involve sentiment analysis, topic modeling, and entity recognition. Sentiment analysis, for example, assigns a numerical value to the emotional tone of a text, allowing for the quantification of public opinion. Topic modeling identifies recurring themes and patterns within the data, revealing underlying trends and narratives.

These extracted features are then incorporated into predictive models, alongside traditional numerical data. Machine learning algorithms, such as recurrent neural networks (RNNs) and transformer models, are particularly well-suited for this task, as they can capture the temporal dependencies and contextual nuances inherent in text data. The models are trained on historical data, allowing them to learn the relationships between textual features and future outcomes.

The applications of text-driven forecasting are diverse and far-reaching. In finance, it can be used to predict stock market fluctuations based on news sentiment and social media activity. In marketing, it can help anticipate consumer trends and optimize product launches by analyzing customer reviews and social media conversations. In public health, it can be used to track the spread of diseases by monitoring online discussions and news reports. Furthermore, this method is useful in political science, by analyzing social media and news, one can attempt to predict election results, and shifts in public opinion.

However, challenges remain. The sheer volume and complexity of textual data can make processing and analysis computationally intensive. The subjective nature of language and the presence of biases can also introduce inaccuracies into the models. Moreover, the dynamic nature of language requires continuous updates and adaptation of the models.

Despite these challenges, the potential of text-driven forecasting is undeniable. As NLP techniques continue to advance and computational power increases, we can expect to see wider adoption of this approach across various domains. By harnessing the power of language, we can gain a deeper understanding of the world around us and make more informed predictions about the future. Text, once considered a mere byproduct of human communication, is now emerging as a powerful tool for unlocking the secrets of prediction.

26 December 2024

How Dramas Achieve Emotional Engagement?

Drama shows are basically idealized versions of reality that are designed for entertainment value and emotional engagement. This is how they achieve engagement.

Exaggeration for Effect: Amplification of emotions, conflicts, and plots to create a captivating narrative. The heightened sense of emotions create larger-than-life characters with over dramatized coincidences.
Simplified Conflicts: While real life problems are more complex, dramas depict simplified versions of conflicts and resolutions in order for the audience to find it easier to follow and understand.
Focus on Entertainment: Focus is on entertainment which prioritizes engaging storylines, memorable characters, satisfactory plot lines and resolutions.
Time Constraints: Storylines are fast paced due to limited budgets with a timeframe.
Idealized Portrayals: Furthermore, idealized versions of relationships, careers, lifestyles add to the dreamy but life-like feel to dramas.
Creative License: There is creative freedom to shape the narrative and character dynamics to articulate the story even if it veers away from reality.

29 November 2023

Recruitment Processes

Making job applications can be frustrating. No one has the time to tailor their resume to each role. And, what is worse is that the recruiter is likely to only be hunting for keywords. If they use an application tracking system it will likely have its own approach to matching. The hiring manager will have their own needs. Most companies have a failed recruitment process where many able candidates are rejected through inadequate recruiters and systems. Automation does not help candidates. However, everyone needs to be given a fair chance of review. Unprofessionalism is typical in most recruitment processes. What is worse is when a candidate goes through multiple stages and towards the end the employer is unable to get funding approval for new hires. Not only is it unfair and frustrating for candidates but it also wastes time on both sides. Streamlining recruitment processes is important for every responsible organization as it is the first point of contact for individuals. Bad recruitment processes is a negative reflection of an organizational practices and the way they treat customers. If the individual has a bad experience as a candidate, they will likely be less inclined to join the company as an employee. All in all, bad recruitment processes can affect the reputation of an organization.

16 May 2023

19 December 2022

Resting Bitch Face

An interesting area to tackle with deep learning for computer vision is in psychology to discover human state of emotions. One such area is RBF - Resting Bitch Face. This facial expression is often characterised with commonality between males and females where they appear unintentionally to display feelings of anger, despair, grumpiness, annoyance, or irritability. However, the state of the individual is likely confused for relaxation or resting. There are often more underlining psychological aspects at play here both from the on looker standpoint as well as from the one projecting such a state of emotion. The phenomenon seems to be fairly a common condition. It often also gets used in fashion and lifestyle photography. Such applications can use methods of stable diffusion either to enhance the condition or further to distort such an appearance. Furthermore, one could develop an emotion model for psychology to study the various facial expressions to understand the mental state of an individual.

9 October 2022

Fifa Qatar 2022

17 June 2022

Neosemantics

16 June 2022

ML is not the answer to Advanced AI

Machine learning by its very nature is built on statistics. If we are to advance AI we have to think beyond machine learning. Humans rarely ever use statistics in the every day life and still have advanced mechanisms to learn through experiences. In fact, those experiences are also retained in memory to form new patterns of learning. Everyday as humans we form associations and relationships with the things around us as we form new experiences. Machine learning on the other hand still requires a lot of training data and that data has to be balanced between variance and bias. Transfer learning on unseen and untrained data is still a challenge. Architectures of deep learning can be formed into very complex and sophisticated structures. However, this complexity is unsustainable when compared to the prohibitive cost and the returns achieved in the process. AI is still very narrow and focused. Any general AI will require thinking beyond the standard concepts of probabilities in statistics. In fact, AI is not just about machine learning but almost eighty percent of the field is based on computer science concepts. The only way to really approach Advanced AI is to take inspiration from the human mind and brain and build models that are highly complex and yet cheaper to put together as building blocks of conceptualization in a hybrid system. Such systems may even be sub-divided into sub-systems just like the organs of the human body and parts of the brain. A natural progression of AI is to combine knowledge representation and reasoning with probabilistic methods to provide such metaphors of adaptability in generalizable learning. Probabilities is not the answer to understanding emotions or other generalizable forms of human learning which lead to brittle and ridged models not to mention a significant margin of error. Machine learning does not provide assurances for key AI functions, which in most cases blurs the lines between what a machine is able to comprehend as a false sense of articulation. For AI to be truely autonomous and live among humans the learning process not only needs to take ethics into account but also be able to reduce such margins of error on its own through the learning process. Increasingly, reinforcement learning methods are being used that do not require huge amounts of training data. However, even in this process learning can be initially slow and also lead to incorrect training in feedback loops which can be disastrous for critical environments like medicine or autonomous driving. An interpretable representation of knowledge is needed to define context as well as some form of logical reasoning constructs. Going further, a long-term and short-term retention of memory through every iterative process of learning is necessary in order to learn from mistakes and past experiences. It may be plausible to assume that to mimic the nervous system one could use more of the statistical thinking to replicate the concepts of impulse and the human senses. Advancement of AI then becomes a joint effort of advancing hardware as well as software. Hardware may even take the form of naturally-inspired computation to enhance the level of coding of information. AI still has a long way to go yet to be regarded as a sentinent being that can cohabitate, live safely among humans, or even to surpass into superintelligence.

11 June 2022

Nova Annotation Tool

2 June 2022

The Dog That Stepped On A Bee

This story is about a victim and an abuser. For some odd reason, society expects the man to always play the role of abuser, while the woman is expected to play the role of the victim. However, in modern day society women neither want to be seen as victims nor do they want men to be chivalrous. They want the legal system to always side with women whether it were wrong or right - to throw away the burden of proof. Whatever happened to the concept of innocent until proven guilty is a typical question one asks when the biased media is seen jumping to conclusions. Since when has the "metoo" movement just been about women? The clash of genders seems like women are all too confused about the definition of equality. Or, is it the fact they want to play the victim card when it suits them? Is the "metoo" movement more about "metoohatemen" movement? Equality is when one sees things beyond the biases of gender stereotypes. However, with equality of opportunity also comes equal levels of consequences for punishment and responsibility for actions. There should be no cause of special treatment for women just because they are women. There should be no drum down special treatment for abusers just because they are women and have softer hands or that they are smaller in stature than men. In the trial for Depp vs Heard, we see what society would define as a type of anomaly of reversed gender roles between victim and abuser. In fact, this trial gives a confident voice to male victims of domestic violence who are often laughed off by women. And, here in lies the truth. To seek the truth one must look beyond the biases and through the spectrum of evidence. Only then can one seek justice and equality. Women should acknowledge when they are wrong and learn to take responsibility for their actions just like men. When one takes an oath, they should uphold that oath to their testimony. Over the years we have seen a shift in the legal system that has gradually sided with women over men in matters of divorce, domestic violence, harassment, abuse, abortion, child custody, and discrimination. But, this only implies that no justice is truly blind. Are our legal courts becoming increasingly biased in their sentences and accountability for justice? Perhaps, it begs one to ponder on the philosophical question of whether justice is a vice or a virtue.

22 May 2022

CMU-MOSEI

9 April 2022

OpenSmile

21 December 2019

Recruitment Agencies

There are some glaring practices in recruitment industry. Some agencies actively practice reverse discrimination by only hiring women for roles such as nurse, personal assistant, receptionist, secretary, catwalk model, and others. However, if the tables were turned and if an agency were to only hire men there would likely be an uproar. This could equally be construed as sexism on part of the organizations that these recruitment agencies are working with to fill such roles. In a society where there is now apparently 100+ genders, how does one even know that the person is a man or a woman anymore? Would a transwoman/transsexual be rejected from such agency for employment? And, should one even care what gender they are? Institutional racism is also a major issue where the recruiter tends to pick candidates based on their inherent bias. Often under the covers the practice continues in some companies in form of cultural fit as an umbrella term. In fact, it doesn't just stop at gender and race. In many cases, it emerges there are all forms of hypocrisy that makes recruitment a very humanly flawed discipline. Also, how can a recruiter profile a candidate purely on basis of a phone and email conversation? And, it is practically impossible for a recruiter to be fully aware of all the skills listed on a person's resume. Such practices have a very dampening effect on the economic demand and supply for jobs and candidates in the market. AI certainly is the way to go to avoid such recruitment risks by retargeting on the things that really matter in employment which is more around the fairness for qualifications, practical experience, and skills to conduct one's job rather than to pass a judgement on their likability factor. However, an important aspect here is that AI should not only be approached via probabilistic methods. Just imagine what would happen if one were to probabilistically cluster bias based on someone's name in order to identify their ethnicity. Can you really tell someone's background based on their name alone? What if they are of mixed background? Won't they be an outlier? AI combines both logical reasoning as well as probabilistic methods.

20 May 2019

GOT Countdown

25 December 2018

MemeTrackers

Memetracker
Techmeme
Memeorandum
knowyourmeme
memeburn
memebuster
nifty

26 January 2018

Spacy.io

15 July 2017

Chatbots Comparative Table

13 July 2017

Fake News Detection

Datasets:
2016-10-facebook-fact-check
kaggle-fake-news
BuzzFeedNews-everything
liar_dataset
politifact-v2apidoc

Fact Checking Sites:
fullfact.org
politifact
opensecrets
snopes
truthorfiction
hoaxslayer
factcheck
Wikipedia
rationalwiki

Reuters Reports:
The Rise of Fact Checking Sites in Europe

Challenges:
Fake News Challenge
Rumor (Pheme)

Sources:
GDELT
Event Registry
SenticNet
ConceptNet

Word Embedding Training Sources:
CommonCrawl
Gigaword
Wikipedia
ConceptNet
SenticNet

Types of Biases:

bias - cognitive - anchoring

bias - cognitive - apophenia

bias - cognitive - attribution

bias - cognitive - confirmation

bias - cognitive - framing

bias - cognitive - halo effect

bias - cognitive - horn effect

bias - cognitive - self-serving

bias - cognitive - status quo

bias - conflict of interest - bribery

bias - conflict of interest - favortism

bias - conflict of interest - funding

bias - conflict of interest - lobbying

bias - conflict of interest - regulatory issues

bias - conflict of interest - shilling

bias - contextual - academic

bias - contextual - educational

bias - contextual - experimenter

bias - contextual - full text on net

bias - contextual - media

bias - contextual - publication

bias - contextual - reporting bias

bias - media - advertising

bias - media - concision

bias - media - corporate

bias - media - coverage

bias - media - false balance

bias - media - gatekeeping

bias - media - mainstream

bias - media - sensationalism

bias - media - statement

bias - media - structural

bias - prejudice/cultural - classism

bias - prejudice/cultural - lookism

bias - prejudice/cultural - racism

bias - prejudice/cultural - sexism

Types of Fake Content:

accounts

bias - cognitive

bias - conflict of interest

bias - contextual

bias - extreme bias

bias - media

bias - prejudice

bias - statistics

claim - cause/effect

claim - definition

claim - extreme claim

claim - fact

claim - policy

claim - value

clickbait - extremebait

clickbait - headlines

clickbait - linking

conspiracy

credibility

deception

fabricated content

false connection

false context

frequency heuristics

gossip

groups

hate speech

hoaxes

imposter

imprecision

influence

irony

junkscience

manipulated content

misleading content

misuse data

parody

partisanship

plagiarized

poll

poor journalism

proceedwithcaution

profit

propaganda

propagation

provoke

repressive state

reviews

rumor

sarcasm

satire

sentiments

source

spam