Analyzing Forex Trading Related Tweets With The Use of NEO4J

The first step in designing a Graph-based database is to create a Concept Map. Similar to relational databases, the concept map acts as the role of entity-relationship diagram for us.

A concept map is a diagram or graphical tool that visually represents relationships between concepts and ideas. Most concept maps depict ideas as boxes or circles (also called nodes), which are structured hierarchically and connected with lines or arrows (also called arcs). These lines are labeled with linking words and phrases to help explain the connections between concepts.

For more information on concept maps use this link:

https://www.lucidchart.com/pages/concept-map/#discovery__top

To create a simple concept map, we divided the tweets into four significant areas (person, news, types, reports) that increase volatility in the forex market.

At the next level under “person”, we have 3 different types of persons that mainly affect market prices; their ideas or analysis could make a significant wave or viewpoint on the whole market and affect traders. The political role has the most effect on the market among this group.

The news could also heavily and harshly affect the volatility and liquidation of the market, and types are those significant items that traders buy or sell in the market, and it reduces to the most important one. For instance, gold price affected USDJPY, or war could affect oil, gold, USDJPY, and USDCAD. There are several complicated relationships between the sub-categories, but due to reduced complexity, they are not mentioned in the concept.

Types have also inbounded and outbound relationships. For instance, silver price effects on NZDUSD or “crude oil inventories” affect “crude oil” and EURUSD.

The reports are divided into two medium and high. We have low importance too, but we did not mention it here due to reducing complexity. We could consider reports as news, but due to reports announced by governmental organizations and having the exact time, I prefer to group them in another category, and news here is described as unpredictable news.

In this part please review this post for a quick review of extracting the tweets with a developer account.

After extracting the tweets we could insert them either manually or by connecting to a database. here is an example of how to insert them manually.

Create (T1: Tweet{id: 11, text: 'Ordinary US people & the media adulating American billionaires is akin to ordinary Russian citizens worshipping the same oligarchs that are fleecing their country blind! #ElonMusk #JeffBezos #BillGates ', URLs: 'https://tinyurl.com/2kbyrarz', createdAt: '1646978400', created_str: '2022-03-08'})
Create (T2: Tweet{id: 12, text: 'Biden says he cant do much about skyrocketing gas prices, shifts blame on Russia #CrudeOil #WW3', location: 'USA', URLs: 'https://tinyurl.com/yckkzrck', createdAt:'1646846340', created_str: '2022-03-09'})
Create (T3: Tweet{id: 13, text: 'crypto world finally become a reality?, Click to know more https://bit.ly/2DPuY8d. #bitcoin #capitalmarket', URLs: 'https://tinyurl.com/bdd3bnvu', createdAt:'1575453300', created_str: '2019-12-04'})
Create (T4: Tweet{id: 14, text: ' Retail business grows 10 pc in February this year: RAI #capitalmarket #Inflation #COVID19 #CrudeOil', location: 'USA', URLs: 'https://tinyurl.com/59f6vk39', createdAt: ' 1647245550', created_str: '2022-03-14'}) Create (T5: Tweet{id: 15, text: '#Gold is down $60, back below $2K per ounce. Since gold stocks barely rose yesterday and are getting clobbered today, gold stocks are now much cheaper than they were on Monday despite a gold price thats basically flat. Gold isnt rallying on #Russia, but #inflation. Buy the dip!', URLs: 'https://tinyurl.com/44nzszx5', createdAt: '1646834160', created_str: '2022-03-09'})
Create (T6: Tweet{id: 16, text: 'More sources are starting to understand the bigger picture. The collapse of the financial system is close and the rebirth is centered around crypto & CBDCs. #sanction #russia #bitcoin #gold #eurusd', location: 'Russia', URLs: 'https://tinyurl.com/mr3k3way', createdAt: '1647198180', created_str: '2022-03-13'})
Create (T7: Tweet{id: 17, text: 'Apparently we are just suppose to smile and thank our government for this opportunity to pay such high prices at the pump. $4.50 here in MI. This is freedom…I’m told. This is America…I’m told. #CrudeOil #inflation #war', URLs: 'https://tinyurl.com/2p8fdjy3', createdAt: '1647272580', created_str: '2022-03-14'})
Create (T8: Tweet{id: 18, text: 'Tesla and SpaceX’s #ElonMusk challenges Russian President Vladimir #Putin to a “single combat” for Ukraine in a tweet.', URLs: 'https://tinyurl.com/2p8zedmy', createdAt: '1647265980', created_str: '2022-03-14'})
Create (T9: Tweet{id: 19, text: 'Every day, in every way, its clearer and clearer that Johnson is the Prime Minister that #Putin put in office in the UK. #BorisJohnson must go. #sanction', location: 'UK', URLs: 'https://tinyurl.com/yckzujs5', createdAt: '1647272400', created_str: '2022-03-14'})
Create (T10: Tweet{id: 20, text: '30 years of Warning of NATO expansion from Putin to the US scholars. The truth has started to come out. #Putin #Nato #Russia', URLs: 'https://tinyurl.com/4anjjs43', createdAt: '1646589960', created_str: '2022-03-06'})

Creating Users, Hashtags, Concept Map Modules as Entities and Relationships in the Graph

#Users
CREATE (user1:User{id: 'Michell06017336'})CREATE (user1)-[:TWEETED {id: 'Michell06017336'}]->(T1)
CREATE (user2:User{id: 'IcanArgue'})CREATE (user2)-[:TWEETED {id: ' IcanArgue'}]->(T2)
CREATE (user3:User{id: 'Televisory_fin'})CREATE (user3)-[:TWEETED {id: 'Televisory_fin'}]->(T3)
CREATE (user4:User{id: 'ETRetail'})CREATE (user4)-[:TWEETED {id: 'ETRetail'}]->(T4)
CREATE (user5:User{id: 'PeterSchiff '})CREATE (user5)-[:TWEETED {id: 'PeterSchiff '}]->(T5)
CREATE (user6:User{id: 'UtilityFTW'})CREATE (user6)-[:TWEETED {id: 'UtilityFTW '}]->(T6)
CREATE (user7:User{id: 'bakerzach922'})CREATE (user7)-[:TWEETED {id: 'bakerzach922'}]->(T7)
CREATE (user8:User{id: 'AlArabia_Eng'})CREATE (user8)-[:TWEETED {id: 'AlArabia_Eng'}]->(T8)
CREATE (user9:User{id: 'Britain_People'})CREATE (user9)-[:TWEETED {id: 'Britain_People'}]->(T9)
CREATE (user10:User{id: 'HahnPhan'})CREATE (user10)-[:TWEETED {id: 'HahnPhan'}]->(T10)

#Hashtags and Their Relationships

CREATE (Hash1: Hash {name: 'bitcoin'})CREATE (Hash2: Hash {name: 'Putin'})CREATE (Hash3: Hash {name: 'ElonMusk'})CREATE (Hash4: Hash {name: 'sanction'})CREATE (Hash5: Hash {name: 'CrudeOil'})CREATE (Hash6: Hash {name: 'capitalmarket'})CREATE (Hash7: Hash {name: 'inflation'})CREATE (Hash8: Hash {name: 'gold'})
CREATE (Hash1)-[:HASH_TAG { name : 'bitcoin' }]->(T6)CREATE (Hash1)-[:HASH_TAG { name : 'bitcoin' }]->(T3)
CREATE (Hash2)-[:HASH_TAG { name : 'putin' }]->(T8)CREATE (Hash2)-[:HASH_TAG { name : 'putin' }]->(T9)CREATE (Hash2)-[:HASH_TAG { name : 'putin' }]->(T10)
CREATE (Hash3)-[:HASH_TAG { name : 'ElonMusk' }]->(T1)CREATE (Hash3)-[:HASH_TAG { name : 'ElonMusk' }]->(T8)
CREATE (Hash4)-[:HASH_TAG { name : 'sanction' }]->(T6)CREATE (Hash4)-[:HASH_TAG { name : 'sanction' }]->(T9)
CREATE (Hash5)-[:HASH_TAG { name : 'CrudeOil' }]->(T2)CREATE (Hash5)-[:HASH_TAG { name : 'CrudeOil' }]->(T4)CREATE (Hash5)-[:HASH_TAG { name : 'CrudeOil' }]->(T7)
CREATE (Hash6)-[:HASH_TAG { name : 'capitalmarket' }]->(T3)CREATE (Hash6)-[:HASH_TAG { name : 'capitalmarket' }]->(T4)
CREATE (Hash7)-[:HASH_TAG { name : 'inflation' }]->(T4)CREATE (Hash7)-[:HASH_TAG { name : 'inflation' }]->(T5)CREATE (Hash7)-[:HASH_TAG { name : 'inflation' }]->(T7)
CREATE (Hash8)-[:HASH_TAG { name : 'gold' }]->(T5)
CREATE (Hash8)-[:HASH_TAG { name : 'gold' }]->(T6)

#Concepts extracted from Concept Map and Their Relationships
CREATE (C1: Concept{name: 'Oil Price'})CREATE (C2: Concept{name: 'Gold Price'})CREATE (C3: Concept{name: 'Retail Sales'})CREATE (C4: Concept{name: 'United States'})CREATE (C5: Concept{name: 'Cryptocurrency'})CREATE (C6: Concept{name: 'Sanctions'})CREATE (C7: Concept{name: 'Elon Musk'})CREATE (C8: Concept{name: 'Vladmir Putin'})
CREATE (C1)-[:CONCEPT { name : 'Oil Price' }]->(T1)CREATE (C1)-[:CONCEPT { name : 'Oil Price' }]->(T2)CREATE (C1)-[:CONCEPT { name : 'Oil Price' }]->(T4)CREATE (C1)-[:CONCEPT { name : 'Oil Price' }]->(T7)
CREATE (C2)-[:CONCEPT { name : 'Gold Price' }]->(T1)CREATE (C2)-[:CONCEPT { name : 'Gold Price' }]->(T2)CREATE (C2)-[:CONCEPT { name : 'Gold Price' }]->(T5)CREATE (C2)-[:CONCEPT { name : 'Gold Price' }]->(T6)CREATE (C2)-[:CONCEPT { name : 'Gold Price' }]->(T7)
CREATE (C3)-[:CONCEPT { name : 'Retail Sales' }]->(T2)CREATE (C3)-[:CONCEPT { name : 'Retail Sales' }]->(T4)
CREATE (C4)-[:CONCEPT { name : 'United States'}]->(T1)CREATE (C4)-[:CONCEPT { name : 'United States'}]->(T2)CREATE (C4)-[:CONCEPT { name : 'United States'}]->(T4)CREATE (C4)-[:CONCEPT { name : 'United States'}]->(T5)CREATE (C4)-[:CONCEPT { name : 'United States'}]->(T7)CREATE (C4)-[:CONCEPT { name : 'United States'}]->(T8)
CREATE (C5)-[:CONCEPT { name : 'Cryptocurrency'}]->(T3)CREATE (C5)-[:CONCEPT { name : 'Cryptocurrency'}]->(T6)
CREATE (C6)-[:CONCEPT { name : 'Sanctions'}]->(T6)CREATE (C6)-[:CONCEPT { name : 'Sanctions'}]->(T9)CREATE (C6)-[:CONCEPT { name : 'Sanctions'}]->(T10)
CREATE (C7)-[:CONCEPT { name : 'Elon Musk'}]->(T1)CREATE (C7)-[:CONCEPT { name : 'Elon Musk'}]->(T3)CREATE (C7)-[:CONCEPT { name : 'Elon Musk'}]->(T8)
CREATE (C8)-[:CONCEPT { name : 'Vladmir Putin'}]->(T1)CREATE (C8)-[:CONCEPT { name : 'Vladmir Putin'}]->(T2)CREATE (C8)-[:CONCEPT { name : 'Vladmir Putin'}]->(T8)CREATE (C8)-[:CONCEPT { name : 'Vladmir Putin'}]->(T9)CREATE (C8)-[:CONCEPT { name : 'Vladmir Putin'}]->(T10)

Now we have our Graph database and we could mover forward and use it for social media analytics.

For instance we want to check our concept elements' priority and check which node has more relationship power in our concept map.

Using this snippet:

MATCH (n)-[r:CONCEPT]->()RETURN r.name, count(*) order by count(*) desc;

Checking with the most central node in the concept map:

Centrality: https://en.wikipedia.org/wiki/Centrality

CALL gds.pageRank.stream({ nodeProjection: "*", relationshipProjection: { LINKED_TO: { type: "*", orientation: "UNDIRECTED", aggregation: "SINGLE" } } }) YIELD nodeId, score RETURN gds.util.asNode(nodeId).id AS ID, score ORDER BY score DESC

List of all tweets associated with the most central node in the concept map:

CALL gds.pageRank.stream({ nodeProjection: "*", relationshipProjection: { LINKED_TO: { type: "*", orientation: "UNDIRECTED", aggregation: "COUNT" } } })YIELD nodeId, score
with gds.util.asNode(nodeId).id AS nodeORDER BY score DESCLIMIT 1match (nodeId) - [:HASH_TAG] ->(p)return p;

List the concepts related to tweets with a particular hashtag:

MATCH (n:Hash)-[r:HASH_TAG {name: 'inflation'}]->(p:Tweet)with pmatch(c:Concept)-[j:CONCEPT]->(p)return collect(distinct j.name)

Page updated

Google Sites

Report abuse