Outlier
Outliers123456import numpy as npimport pandas as pdimport matplotlib.pyplot as pltimport seaborn as snspd.set_option("display.notebook_repr_html", False) # disable "rich" outputplt.style.use("seaborn") Unidimensional Data1234567x = np.loadtxt("https://raw.githubusercontent.com/gagolews/" + "teaching-data/master/marek/blobs2.txt")plt.subplot(121)sns.boxplot(data=x, orient="h")plt.subplot(122)sns.histplot(x, binwidth=1)plt.show() M...
Continuous Probability Distributions
Continuous Probability Distributions1234import numpy as npimport matplotlib.pyplot as pltimport seaborn as snsplt.style.use("seaborn") 123456heights = np.loadtxt("https://raw.githubusercontent.com/gagolews/" + "teaching-data/master/marek/nhanes_adult_female_height_2020.txt")sns.histplot(heights, stat="density", kde=True)plt.show()import scipy.stats Normal DistributionNormal Distribution12345678μ = np.mean(heights) # an estimator of expected valueσ =...
Multivariate Categorical and Relational Data
Multivariate Categorical and Relational Data12345import numpy as npmarathon = np.loadtxt("https://raw.githubusercontent.com/gagolews/" + "teaching-data/master/marek/37_pzu_warsaw_marathon_3groups_top1000.txt", delimiter=",", dtype=str)marathon[:6, :] # preview Two-Way Contingency Tables123np.unique(marathon[:, 0])np.unique(marathon[:, 1]) 123456import scipy.statsl, v = scipy.stats.contingency.crosstab(marathon[:, 0], marathon[:, 1])l, vimport marekmarek.pr...
Visualising Multidimensional Data and Measuring Correlation
Visualising Multidimensional Data and Measuring Correlation123456789import numpy as npimport pandas as pdbody = pd.read_csv("https://raw.githubusercontent.com/gagolews/" + "teaching-data/master/marek/nhanes_adult_female_bmx_2020.csv", comment="#")body = body.to_numpy() # data frames will be covered laterbody.shapebody[:6, :] # 6 first rows, all columns Scatterplots2D Data1234567891011import matplotlib.pyplot as pltimport seaborn as snsplt.style.use("...
Handling Categorical Data
Handling Categorical DataRepresenting Categorical DataTwo common ways to represent a categorical variable with k distinct levels is by storing it as: a vector of strings, a vector of integers between 0 (inclusive) and k (exclusive). 12345678import numpy as npcountries = np.loadtxt("https://raw.githubusercontent.com/gagolews/" + "teaching-data/master/marek/37_pzu_warsaw_marathon_country.txt", dtype="str")x = countries[:16]xnp.unique(x) Encoding and Decodin...
Inspecting the Distribution of Numberic Data
Inspecting the Distribution of Numberic Data12345import numpy as npheights = np.loadtxt("https://raw.githubusercontent.com/gagolews/" + "teaching-data/master/marek/nhanes_adult_female_height_2020.txt")np.random.choice(heights, 24, replace=False) Histograms1234567import matplotlib.pyplot as pltimport seaborn as snsplt.style.use("seaborn")sns.__version__ # FYIsns.histplot(heights, bins=11)plt.show() 1234income = np.loadtxt("https://raw.githubusercontent.co...
Descriptive Statistic for Continuous Data
Descriptive Statistic fopr Continuous DataHistograms are based on binned data and hence provide us with snapshots of how much probability mass is allocated in diferent parts of the data domain. 1234567import numpy as npincome = np.loadtxt("https://raw.githubusercontent.com/gagolews/" + "teaching-data/master/marek/uk_income_simulated_2020.txt")b = [0, 10000, 20000, 30000, 40000, 50000, 60000, 80000, np.inf] # bin boundsc = np.histogram(income, bins=b)[0] # countsfor i ...
Python - Basic Usage of Data Analysis
Basic Usage of Data AnalysisGetting Started with JupyterLabJupyterLab is a web-based development environment supporting numerous programming languages, including, of course, Python. jupyterlab Scalar Types in PythonBasic Operations on Data Framesdoc pre-requisite123import numpy as npimport pandas as pdpd.set_option("display.notebook_repr_html", False) # disable "rich" output Aggregating1234567891011121314151617181920np.random.seed(123)d = pd.DataFrame(dict( u = np.roun...
Hello Prisma
Connecting Prisma application to Supabase PostgresCreste a custom user fro Prismacreate a Prisma DB user with full privileges on the public schema123456789101112131415-- Create custom usercreate user "prisma" with password 'custom_password' bypassrls createdb;-- extend prisma's privileges to postgres (necessary to view changes in Dashboard)grant "prisma" to "postgres";-- Grant it necessary permissions over the relevant schemas (public)grant usage o...