FinanceDatabase: A database of 300.000 symbols (ETFs, Currencies, Crypto)

As a non-public investor, the sheer quantity of files that might perhaps presumably also moreover be stumbled on on the bag is extremely daunting. Making an are trying to
understand what form of corporations or ETFs are on hand is extremely great with there being tens of millions of
corporations and derivatives on hand available on the market. Sure, basically the most traded corporations and ETFs can mercurial be stumbled on
fair this capability that of they are known to the general public (for instance, Microsoft, Tesla, S&P500 ETF or an All-World ETF). Then all but again,
what else is available is frequently unknown.

This database tries to clear up that. It functions 300.000 symbols containing Equities, ETFs, Funds, Indices,
Currencies, Cryptocurrencies and Money Markets. It subsequently helps you to form a huge overview of sectors,
industries, forms of investments and great more.

The goal of this database is explicitly not to offer up-to-date fundamentals or stock files as these might perhaps presumably also moreover be obtained
with ease (with the help of this database) by utilizing yfinance,
FundamentalAnalysis or
ThePassiveInvestor. As an various, it affords insights into the merchandise
that exist in each country, commerce and sector and affords basically the most wanted files about each product. With
this files, that you simply can analyse particular areas of the financial world and/or procure a product that is arduous to search out.
Look for examples on the vogue that you simply can mix this database, and the earlier talked about programs the allotment
Examples.

Some key statistics of the database:

Product	Quantity	Sectors	Industries	Worldwide locations	Exchanges
Equities	155.705	16	242	111	82
ETFs	36.727	364*	94*	100	52
Funds	57.816	1678*	438*	100	34

Product	Quantity	Class
Currencies	2.590	174 Currencies
Cryptocurrencies	3.624	299 Cryptocurrencies
Indices	86.353	49 Exchanges
Money Markets	1.384	2 Exchanges

These numbers discuss about with families (iShares, Vanguard) and classes (World Stock, Right Property) respectively.
Here is an estimation. Obtaining the country distribution can handiest be performed by accumulating files on the underlying
or by handbook search.

Desk of Contents

The consume of the Database

To entry the database that you simply can download the total repository, however I strongly imply making consume of the bundle
intently attached to the database. It helps you to make a preference particular json files as properly as search by composed
files with a particular ask.

Set up

That you simply can presumably install the bundle with the next steps:

pip install financedatabase
- Alternatively, download the ‘Searcher’ itemizing.
(within Python) import financedatabase as fd

Functions

The bundle has the next functions:

show_options(product, equities_selection=None, country=None, sector=None, commerce=None) – affords all on hand solutions from
the functions below per product (i.e. Equities, Funds) which then might perhaps presumably also moreover be historical to receive files. That you simply can presumably capture a
sub assortment of equities by getting into ‘countries’, ‘sectors’ or ‘industries’ for equities_selection as properly as capture
the recount sectors and industries per country and industries per sector by the ‘country’ and ‘sector’ parameters.
The commerce variable is a boolean that returns all industries, to what sector each corresponds and all countries
which like corporations on this commerce.
select_cryptocurrencies(cryptocurrency=None) – with out a enter affords all cryptocurrencies, with enter affords
the cryptocurrency of preference.
select_currencies(forex=None) – with out a enter affords all currencies, with enter affords
the forex of preference.
select_etfs(category=None) – with out a enter affords all etfs, with enter affords all etfs of a
particular category.
select_equities(country=None, sector=None, commerce=None) – with out a enter affords all equities, with enter
affords all equities of a rustic, sector, commerce or a aggregate of the three.
select_funds(category=None) – with out a enter affords all funds, with enter affords all funds of a
particular category.
select_indices(market=None) – with out a enter affords all indices, with enter affords all indices of a
particular market which on the full refers to indices in a particular country (admire de_market affords DAX).
select_moneymarkets(market=None) – with out a enter affords all moneymarkets, with enter affords all moneymarkets of a
particular market which on the full refers to moneymarkets in a particular country.
search_products(database, ask, search='abstract', case_sensitive=False, new_database=None) – with enter
from the above functions, this characteristic searches for particular values (i.e. the ask ‘sustainable’) in
regarded as one of many keys of the dictionary (which is by default the abstract). It also has the option to allow
case-composed procuring which is off by default.

For users of the dealer DeGiro, that you simply would per chance presumably be in a utter to search out files on the tickers stumbled on in the
Fee Free ETFs list by selecting either
core_selection_degiro_filled (all files) or core_selection_degiro_filtered (filtered by abstract) as category
when utilizing the characteristic select_etfs.

By default, exchanges will not be integrated in the preference functions. Therefore, the quantity of files returned is less
than depicted in basically the major statistics. Within the occasion you make a selection to encompass all exchanges, please utter exclude_exchanges to False.

For added files about each characteristic that you simply can consume the fabricate-in help characteristic of Python. For
instance help(show_options) returns a conventional description, the that that you simply can imagine enter parameters and what is returned
as output.

Evolved Usage

Within the occasion you make a selection to store the database at a certain device (for instance your bag Fork) that you simply can create so with the variable
base_url which yow will detect in each of the above ‘capture’ functions. An instance can be:

select_funds(category='Africa Equity', base_url=)

That you simply can presumably also store the database in the neighborhood and show your local device with the variable base_url and by setting
use_local_location to Accurate. An instance can be:

select_etfs(category='Bank Mortgage', base_url='C:/Customers/jerbo/FinanceDatabase/Database/ETFs/', use_local_location=Accurate)

Examples

This allotment affords just a few examples of the possibilities with this bundle. These are merely just some of the things you
can create with the bundle. As that you simply can form a huge vary of symbols, elegant great any
bundle that requires symbols have to composed work.

Firms in the Netherlands

Working out which sectors exist in a rustic might perhaps presumably also moreover be attention-grabbing. No longer handiest to realise the point of hobby of the country however
also to realise which home holds basically the most files. Here is an indication of the show_options characteristic.
A characteristic principal to querying files from the Database.

Let’s originate by acquiring the outlandish countries, sectors and industries of all equities in the database:

import financedatabase as fd
# Receive all countries from the database
equities_countries=fd.show_options('equities', 'countries')
# Receive all sectors from the database
equities_sectors=fd.show_options('equities', 'sectors')
# Receive all industries from the database
equities_industries=fd.show_options('equities', 'industries')
# Receive all countries   sectors   industries from the database
equities_all_categories=fd.show_options('equities')

This affords the next lists (where equities_all_categories is a dictionary with these three lists):

Then, I are searching for to explore how many corporations exist in each sector in the Netherlands. Let’s count all corporations with the
following code, I skip a sector when it has no files and likewise create not encompass corporations which might be not categorized:

equities_per_sector_netherlands={}
for sector in equities_sectors[1:]:
    strive:
        equities_per_sector_netherlands[sector]=len(fd.select_equities(country='Netherlands', sector=sector))
    with the exception of ValueError as error:
        print(error)

Lastly, I device the guidelines in a pie chart and add some formatting to originate the pie chart watch rather nicer:

myth, values=zip(*equities_per_sector_netherlands.devices())
colors=['b', 'g', 'r', 'c', 'm', 'y', 'k', 'tab:blue', 'tab:orange', 'tab:gray',
          'lightcoral', 'yellow', 'saddlebrown', 'lightblue', 'olive']
plt.pie(values, labels=myth, colors=colors,
        wedgeprops={'linewidth': 0.5, 'edgecolor': 'white'})
plt.title('Firms per sector in the Netherlands')
plt.tight_layout()
plt.show()

This ends in the next graph which affords a tag which sectors are dominant within The Netherlands.
Unnecessary to claim this is a mere instance and to in reality understand the significance of obvious corporations for the Netherlands,
an in-depth prognosis have to be performed.

Technical Analysis of Biotech ETFs

With the help of ta and yfinance I will
mercurial invent a widespread technical prognosis on a personnel of ETFs categorized by the FinanceDatabase. I originate by
procuring the database for ETFs linked to Nicely being after which originate a subselection by procuring, in the composed database,
for biotech-linked ETFs:

import financedatabase as fd
health_etfs=fd.select_etfs(category='Nicely being')
health_etfs_in_biotech=fd.search_products(health_etfs, 'biotech')

Then, I receive stock files on each ticker and take away tickers that don’t like any files in my chosen period. The period I even like
chosen displays the preliminary impact of the Coronacrisis on the financial markets.

import yfinance as yf
stock_data_biotech=yf.download(list(health_etfs_in_biotech.keys()), originate="2020-01-01", cease="2020-06-01")['Adj Close']
stock_data_biotech=stock_data_biotech.dropna(axis='columns')

Next up I initialise subplots and loop over all composed tickers. Here, I fabricate a brand unique short-term DataFrame that I possess
with the adjusted finish prices of the ticker as properly because the Bollinger Bands. Then I device the guidelines in regarded as one of
the subplots.

import pandas as pd
from ta.volatility import BollingerBands
import matplotlib.pyplot as plt
make a selection, axis=plt.subplots(4, 3)
row=0
column=0
for ticker in stock_data_biotech.columns:
    data_plot=pd.DataFrame(stock_data_biotech[ticker])
    indicator_bb=BollingerBands(finish=stock_data_biotech[ticker], window=20, window_dev=2)
    data_plot['bb_bbm']=indicator_bb.bollinger_mavg()
    data_plot['bb_bbh']=indicator_bb.bollinger_hband()
    data_plot['bb_bbl']=indicator_bb.bollinger_lband()
    axis[row, column].device(data_plot)
    axis[row, column].set_title(health_etfs_in_biotech[ticker]['long_name'], fontsize=6)
    axis[row, column].set_xticks([])
    axis[row, column].set_yticks([])
    column =1
    if column==3:
        row =1
        column=0
        
make a selection.suptitle('Technical Analysis of Biotech ETFs for the length of Coronacrisis')
make a selection.tight_layout()

This ends in the next graph which affords a tag wether Biotech ETFs had been oversold or overbought and
how this create is neutralised (to some level) in the months after. Read more
about Bollinger Bands right here.

United States’ Airlines

If I make a selection to form all corporations for the length of the US listed below ‘Airlines’ I will write the
following code:

import financedatabase as fd
airlines_us=fd.select_equities(country='United States', commerce='Airlines')

Then, I will consume programs admire yfinance to mercurial receive files from
Yahoo Finance for every symbol in the commerce admire this:

from yfinance.utils import get_json
from yfinance import download
airlines_us_fundamentals={}
for symbol in airlines_us:
    airlines_us_fundamentals[symbol]=get_json("https://finance.yahoo.com/quote/"   symbol)
airlines_us_stock_data=download(list(airlines_us))

With just a few traces of code, I even like composed all files from a particular commerce for the length of the US. From right here on
that you simply can compare elegant great any key statistic, major- and stock files. As an illustration, let’s device a simple bar
chart that affords insights in the Rapidly Ratios (indicator of the overall financial energy or weak point of an organization):

import matplotlib.pyplot as plt
for symbol in airlines_us_fundamentals:
    quick_ratio=airlines_us_fundamentals[symbol]['financialData']['quickRatio']
    long_name=airlines_us_fundamentals[symbol]['quoteType']['longName']
    if quick_ratio is None:
        continue
    plt.barh(long_name, quick_ratio)
plt.tight_layout()
plt.show()

Which ends in the graph displayed below (as of the 18th of October 2021). From this graph that you simply can title
corporations that currently lack ample sources to quilt their liabilities (snappy ratio 1). Both too low and too high might perhaps presumably originate you wonder whether or not the company adequately
manages its sources.

Silicon Valley’s Market Cap

If I are searching for to realise which listed technology corporations exist in Silicon Valley, I will receive all equities of
the sphere ‘Skills’ after which filter basically based on metropolis to form all listed technology corporations in ‘Silicon Valley’.
The metropolis ‘San Jose’ is where Silicon Valley is positioned.

import financedatabase as fd
all_technology_companies=fd.select_equities(sector='Skills')
silicon_valley=fd.search_products(all_technology_companies, ask='San Jose', search='metropolis')

Then I originate accumulating files with the FundamentalAnalysis bundle.
Here I receive basically the major metrics which encompass 57 different metrics (starting from PE ratios to Market Cap).

import FundamentalAnalysis as fa
API_KEY="YOUR API KEY HERE"
data_set={}
for ticker in silicon_valley:
    strive:
        data_set[ticker]=fa.key_metrics(ticker, API_KEY, period='annual')
    with the exception of Exception:
        continue

Then I originate a preference basically based on the final 5 years and filter by market cap to take a look at the corporations by formula of dimension
with one but some other. This also causes corporations which like not been listed for five years to be filtered out of my dataset.
Lastly, I device the guidelines.

import pandas as pd
import matplotlib.pyplot as plt
years=['2016', '2017', '2018', '2019', '2020']
market_cap=pd.DataFrame(index=years)
for ticker in data_set:
    strive:
        data_years=[]
        for 365 days in years:
            data_years.append(data_set[ticker].loc['marketCap'][year])
        market_cap[all_technology_companies[ticker]['short_name']]=data_years
    with the exception of Exception:
        continue
market_cap_plot=market_cap.device.bar(stacked=Accurate, rot=0, colormap='Spectral')
market_cap_plot.myth(prop={'dimension': 5.25})
plt.show()

This ends in the graph displayed below which separates the tiny corporations from the clear corporations. Present that
this does not encompass all technology corporations in Silicon Valley this capability that of most will not be listed or will not be integrated
in the database of the FundamentalAnalysis bundle.

Core Probability ETFs

Usually, Excel simply affords the correct solution whereas you happen to make your mind up on to love compare a vary of ETFs mercurial. Therefore, but some other
option is to consume my program ThePassiveInvestor. The perform of
this program is to mercurial compare a clear assortment of ETFs by accumulating their major attributes
(i.e. holdings, return, volatility, monitoring error).

As I make investments with DeGiro, a colossal originate for me can be by accumulating all ETFs which might be listed for the length of the Core
Probability (price free) list of my dealer with the next code (or manually form them from the json file):

import financedatabase as fd
core_selection=fd.select_etfs("core_selection_filtered", exclude_exchanges=False)

Then I convert the keys of the core_selection into a Sequence and ship it to Excel without index and header.

import pandas as pd
tickers=pd.Sequence(core_selection.keys())
tickers.to_excel('core_selection_tickers.xlsx', index=None, header=None)

Within the occasion you open the Excel file created you explore the next lay-out (which corresponds to the lay-out current
by this system):

Then I open ThePassiveInvestor program and consume the Excel as enter. The major enter is the Excel that you simply make a selection to love to
be stuffed with enter from your tickers (created by this system). The 2nd enter is the file you created above.

Must you bustle this system it begins accumulating files on each ticker and fills the Excel with files. After this system
is completed that you simply would per chance presumably be in a utter to search out an Excel that appears to be very great admire the GIF you explore below. With this files that you simply can
safe a tag whether or not the ETF is what that you simply would per chance presumably be procuring for.

Questions & Solutions

In this allotment yow will detect solutions to steadily asked questions. In case the answer to your ask will not be right here,
take into accout growing an Shriek.

How did you safe your files?
- Please verify the Methodology.
Is there beef up for ?
- Yes, presumably there is because the database involves 111 countries. Please verify
  right here.
How can I procure out which countries, sectors and/or industries exists for the length of the database without desirous to substantiate
the database manually?
- For this that you simply can consume the show_options characteristic from the bundle attached to this database. Please explore
  this instance
As soon as I receive files by yfinance I see that not all tickers return output, why is that?
- Some tickers are merely holdings of corporations and subsequently create not in reality like any files attached to them.
  Therefore, it is far real looking that not all tickers return files. Within the occasion that you simply would per chance presumably be composed in doubt, search the ticker on
  Google to explore if there is in reality no files on hand.
How recurrently does the Database safe up to this point?
- I goal at doing this each few months. The database does not like to safe up to this point recurrently since the guidelines
  composed is handiest traditional files. As an illustration, a Sector title usually changes and corporations create not are inclined to
  cross to but some other country each few months. Therefore, the guidelines have to composed protect up to this point for several months.
  Within the occasion you make a selection to make a contribution to updating the database then that is device appreciated. Please verify the
  Methodology for steerage on how.
Assemble your sector and commerce names consume the identical naming convention as GIC sector?
- No longer fully however very the same, or not it is basically based on Yahoo Finance’s sectors and industries. Search industries and
  sectors. Per chance a future adjustment can be to originate them aligned with GICS.

Contribution

Projects are lunge to love (tiny) errors and might perhaps presumably continuously be improved. Therefore, I highly support you to submit
points and fabricate pull requests to toughen the bundle.

The final replace to the database is the 18th of October 2021. I continuously procure Pull Requests each few months
to protect the database up to this point. Extending the quantity of tickers and files is also great appreciated. Must you make a selection to create
this, please present me first to total a few users doing the right identical part.

>
𝚆𝚊𝚝𝚌𝚑 𝙽𝙾𝚆 📺