edamame.eda

eda

edamame.eda.eda.correlation_categorical(data: DataFrame) → None

The function performs the Chi-Square Test of Independence between categorical variables of the dataset.

Parameters:

data (pd.DataFrame) – A pandas DataFrame passed in input.

Returns:

None

Example

>>> import edamame.eda as eda
>>> df = pd.DataFrame({'category': ['A', 'B', 'A', 'B', 'C', 'A'], 'value': [1, 2, 3, 4, 4, 5], 'category_2': ['A2', 'A2', 'B2', 'B2', 'A2', 'B2']})
>>> eda.correlation_categorical(df)

edamame.eda.eda.correlation_pearson(data: DataFrame, threshold: float = 0.0) → None

The function performs the Pearson’s correlation between the columns pairs.

Parameters:

• data (pd.DataFrame) – A pandas DataFrame passed in input.

• threshold (float) – Only the correlation values higher than the threshold are shown in the matrix. A floating value set by default to 0.

Returns:

None

Example

>>> import edamame.eda as eda
>>> df = pd.DataFrame({'category': ['A', 'B', 'A', 'B', 'C', 'A'], 'value': [1, 2, 3, 4, 4, 5], 'type': [0,1,0,1,0,1]})
>>> num_col, qual_col = eda.variables_type(df)
>>> eda.correlation_pearson(df, num_col)

edamame.eda.eda.correlation_phik(data: DataFrame, theory: bool = False) → None

Paper link: https://arxiv.org/pdf/1811.11440.pdf

Parameters:

• data (pd.DataFrame) – A pandas DataFrame passed in input.

• theory (bool) – A boolean value for displaying insight into the theory of the Phik correlation index. By default is set to False.

Returns:

None

Example

>>> import edamame.eda as eda
>>> df = pd.DataFrame({'category': ['A', 'B', 'A', 'B', 'C', 'A'], 'value': [1, 2, 3, 4, 4, 5], 'category_2': ['A2', 'A2', 'B2', 'B2', 'A2', 'B2']})
>>> eda.correlation_phik(df, theory=True)

edamame.eda.eda.describe_distribution(data: DataFrame) → None

The function display the result of the describe() method applied to a pandas dataframe, divided by numerical and object columns.

Parameters:

data (pd.DataFrame) – A pandas DataFrame passed in input.

Returns:

None

Example

>>> import edamame.eda as eda
>>> df = pd.DataFrame({'category': ['A', 'B', 'A', 'B'], 'value': [1, 2, 3, 4]})
>>> eda.describe_distribution(df)

edamame.eda.eda.dimensions(data: DataFrame) → None

The function displays the number of rows and columns of a pandas dataframe passed.

Parameters:

data (pd.Dataframe) – A pandas DataFrame passed in input.

Returns:

None

Example

>>> import edamame.eda as eda
>>> df = pd.DataFrame({'category': ['A', 'B', 'A', 'B'], 'value': [1, 2, 3, 4]})
>>> eda.dimensions(df)

edamame.eda.eda.drop_columns(data: DataFrame, col: List[str])

The function returns a pandas dataframe with the columns selected dropped.

Parameters:

• data (pd.DataFrame) – A pandas DataFrame passed in input.

• col (List[str]) – A list of strings containing the names of columns to drop.

Returns:

pd.DataFrame

Example

>>> import edamame.eda as eda
>>> df = pd.DataFrame({'category': ['A', 'B', 'A', 'B'], 'value': [1, 2, 3, 4], 'type': [0,1,0,1]})
>>> df = eda.drop_columns(df, col=["category", "type"])

edamame.eda.eda.handling_missing(data: DataFrame, col: List[str], missing_val: float | int = nan, method: List[str] = []) → DataFrame

The function returns a pandas dataframe with the columns selected modified to handle the NaN values. It’s easy to use after the execution of the missing function.

Parameters:

• data (pd.DataFrame) – A pandas DataFrame passed in input.

• col (List[str]) – A list of the names of the dataframe columns to handle.

• missing_val (Union[float, int]) – The value that represents the NA in the columns passed. By default is equal to np.nan but can be set as other value like 0.

• method (List[str]) – A list of the names of the methods (mean, median, most_frequent, drop) applied to the columns passed. By default, if nothing was indicated, the function applied the most_frequent method to all the columns passed. Indicating fewer methods than the names of the columns leads to an autocompletion with the most_frequent method.

Returns:

Return the processed dataframe

Return type:

pd.DataFrame

Example

>>> import edamame.eda as eda
>>> import pandas as pd
>>> import numpy as np
>>> df = pd.DataFrame({'category': ['0', '1', '0', '1', np.nan], 'value': [1, 2, 3, 4, np.nan], 'value_2': [-1,-2,0,0,0]})
>>> nan_quant, nan_qual, zero_col = eda.missing(df)
>>> df = eda.handling_missing(df, col = nan_quant, missing_val = np.nan, method = ['mean']*len(nan_quant)) # handle NaN for numerical columns
>>> df = eda.handling_missing(df, col = nan_qual, missing_val=np.nan, method=['most_frequent']*len(nan_qual)) # handle NaN for categorical columns
>>> df = eda.handling_missing(df, col = zero_col, missing_val=0, method=['mean']*len(zero_col)) # handle O for columns with too many zeros

edamame.eda.eda.identify_types(data: DataFrame) → Tuple[List[str], List[str]]

The function display the result of the dtypes method.

Parameters:

data (pd.DataFrame) – A pandas DataFrame passed in input.

Returns:

A tuple contains a list with the numerical columns and a list with the categorical/object column.

Return type:

Tuple[List[str], List[str]]

Example

>>> import edamame.eda as eda
>>> df = pd.DataFrame({'category': ['A', 'B', 'A', 'B'], 'value': [1, 2, 3, 4]})
>>> quant_col, qual_col = eda.identify_types(data)

edamame.eda.eda.inspection(data: DataFrame, threshold: int = 10, bins: int = 50, figsize: Tuple[float, float] = (6.0, 4.0)) → None

The function displays an interactive plot for analysing the distribution of a variable based on the distinct cardinalities of the target variable.

Parameters:

• data (pd.DataFrame) – A pandas DataFrame passed in input.

• threshold (int) – A value for determining the maximum number of distinct cardinalities the target variable can have. By default is set to 10.

• bins (int) – The number of bins used by the histograms. By default bins=50.

• figsize (Tuple[float, float]) – A tuple to determine the plot size.

Returns:

None

Example

>>> import edamame.eda as eda
>>> df = pd.DataFrame({'category': ['A', 'B', 'A', 'B', 'C', 'A'], 'value': [1, 2, 3, 4, 4, 5]})
>>> eda.inspection(df)

edamame.eda.eda.interaction(data: DataFrame) → None

The function display an interactive plot for analysing relationships between numerical columns with a scatterplot.

Parameters:

data (pd.DataFrame) – A pandas DataFrame passed in input.

Returns:

None

Example

>>> import edamame.eda as eda
>>> df = pd.DataFrame({'category': ['A', 'B', 'A', 'B', 'C', 'A'], 'value': [1, 2, 3, 4, 4, 5]})
>>> eda.interaction(df)

edamame.eda.eda.missing(data: DataFrame) → Tuple[List[str], List[str], List[str]]

The function display the following elements:

• A table with the percentage of NA record for every column.

• A table with the percentage of 0 as a record for every column.

• A table with the percentage of duplicate rows.

• A list of lists that contains the name of the numerical columns with NA, the name of the categorical columns with NA and the name of the columns with 0 as a record.

Parameters:

data (pd.DataFrame) – A pandas DataFrame passed in input.

Returns:

A Tuple that contains the name of the numerical columns with NA, the name of the categorical columns with NA and the name of the columns with 0 as a record.

Return type:

Tuple[List[str], List[str], List[str]]

Example

>>> import edamame.eda as eda
>>> import pandas as pd
>>> import numpy as np
>>> df = pd.DataFrame({'category': ['0', '1', '0', '1', np.nan], 'value': [1, 2, 3, 4, np.nan]})
>>> nan_quant, nan_qual, zero_col = eda.missing(df)

edamame.eda.eda.modify_cardinality(data: DataFrame, col: List[str], threshold: List[int]) → DataFrame

The function returns a pandas dataframe with the cardinalities of the columns selected modified.

Parameters:

• data (pd.DataFrame) – A pandas DataFrame passed in input.

• col (List[str]) – A list of strings containing the names of columns for which we want to modify the cardinalities.

• threshold (List[int]) – A list of integer values containing the threshold values for every variable.

Returns:

pd.DataFrame

Example

>>> import edamame.eda as eda
>>> df = pd.DataFrame({'category': ['A', 'B', 'A', 'B', 'C', 'A'], 'value': [1, 2, 3, 4, 4, 5], 'type': [0,1,0,1,0,1]})
>>> df = eda.modify_cardinality(data_cpy, col = ['category'], threshold=[3])

edamame.eda.eda.num_to_categorical(data: DataFrame, col: List[str]) → DataFrame

The function returns a dataframe with the columns transformed into an “object”.

Parameters:

• data (pd.DataFrame) – A pandas DataFrame passed in input.

• col (List[str]) – A list of strings containing the names of columns to convert.

Returns:

Dataframe with numerical columns passed converted to categorical.

Return type:

pd.DataFrame

Example

>>> import edamame.eda as eda
>>> df = pd.DataFrame({'category': ['0', '1', '0', '1'], 'value': [1, 2, 3, 4]})
>>> df = eda.num_to_categorical(df, col=["category"])

edamame.eda.eda.num_variable_study(data: DataFrame, col: str, bins: int = 50, epsilon: float = 0.0001, theory: bool = False) → None

The function displays the following transformations of the variable col passed: log(x), sqrt(x), x^2, Box-cox, 1/x

Parameters:

• data (pd.DataFrame) – A pandas DataFrame passed in input.

• col (str) – The name of the dataframe column to study.

• bins (int) – The number of bins used by the histograms. By default bins=50.

• epsilon (float) – A constant for handle non strictly positive variables. By default epsilon = 0.0001

• theory (bool) – A boolean value for displaying insight into the transformations applied.

Returns:

None

Example

>>> import edamame.eda as eda
>>> df = pd.DataFrame({'category': ['A', 'B', 'A', 'B', 'C', 'A'], 'value': [1, 2, 3, 4, 4, 5]})
>>> eda.num_variable_study(df, 'value', bins = 50, theory=True)

edamame.eda.eda.plot_categorical(data: DataFrame, col: List[str]) → None

The function returns a sequence of tables and plots.

Parameters:

• data (pd.DataFrame) – A pandas DataFrame passed in input.

• col (List[str]) – A list of string containing the names of columns to plot.

Returns:

None

Example

>>> import edamame.eda as eda
>>> df = pd.DataFrame({'category': ['A', 'B', 'A', 'B'], 'value': [1, 2, 3, 4], 'type': [0,1,0,1]})
>>> num_col, qual_col = eda.variables_type(df)
>>> eda.plot_categorical(df, qual_col)

edamame.eda.eda.plot_numerical(data: DataFrame, col: List[str], bins: int = 50) → None

The function returns a sequence of tables and plots.

Parameters:

• data (pd.DataFrame) – A pandas DataFrame passed in input.

• col (List[str]) – A list of string containing the names of columns to plot.

• bins (int) – Number of bins to use in the histogram plot.

Returns:

None

Example

>>> import edamame.eda as eda
>>> df = pd.DataFrame({'category': ['A', 'B', 'A', 'B'], 'value': [1, 2, 3, 4], 'type': [0,1,0,1]})
>>> num_col, qual_col = eda.variables_type(df)
>>> eda.plot_numerical(df, num_col, bins = 100)

edamame.eda.eda.split_and_scaling(data: DataFrame, target: str, minmaxscaler: bool = False) → Tuple[DataFrame, DataFrame]

The function returns two pandas dataframes:

• The regressor matrix X contains all the predictors for the model.

• The series y contains the values of the response variable.

In addition, the function applies a step of standard scaling on the numerical columns of the X matrix.

Parameters:

• data (pd.DataFrame) – A pandas DataFrame passed in input.

• target (str) – The response variable column name.

• minmaxscaler (bool) – Select the type of scaling to apply to the numerical columns.

Returns:

Return the regression matrix and the target column.

Return type:

Tuple[pd.DataFrame, pd.DataFrame]

Example

>>> import edamame.eda as eda
>>> df = pd.DataFrame({'category': ['A', 'B', 'A', 'B', 'C', 'A'], 'value': [1, 2, 3, 4, 4, 5], 'target': ['A2', 'A2', 'B2', 'B2', 'A2', 'B2']})
>>> X, y = eda.split_and_scaling(df, 'target')

edamame.eda.eda.view_cardinality(data: DataFrame, col: List[str]) → None

The function especially helps study the cardinalities of the categorical variables.

Parameters:

• data (pd.DataFrame) – A pandas DataFrame passed in input.

• col (List[str]) – A list of strings containing the names of columns for which we want to show the number of unique values.

Returns:

None

Example

>>> import edamame.eda as eda
>>> df = pd.DataFrame({'category': ['A', 'B', 'A', 'B'], 'value': [1, 2, 3, 4], 'type': [0,1,0,1]})
>>> num_col, qual_col = eda.variables_type(df)
>>> eda.view_cardinality(df, qual_col)

tools

edamame.eda.tools.dataframe_review(data: DataFrame) → None

The function checks if the object passed is a Pandas dataframe.

Parameters:

data (pd.Dataframe) – A pandas DataFrame passed in input.

Raises:

TypeError – If the input DataFrame contains non-numerical columns.

Returns:

None

edamame.eda.tools.dummy_control(data: DataFrame) → None

The function checks if the Pandas dataframe passed is encoded with dummy or OHE.

Parameters:

data (pd.Dataframe) – A pandas DataFrame passed in input.

Raises:

TypeError – If the input DataFrame contains non-numerical columns.

Returns:

None

edamame.eda.tools.load_model(path: str)

The function load the model saved previously in the pickle format.

Parameters:

path (str) – Path to the model saved in .pkl

edamame.eda.tools.ohe(array: ndarray) → ndarray

Convert a NumPy array that represents the categorical label of the target variable and transform it using one-hot encoding.

Parameters:

array (np.ndarray) – The target variables passed in input.

Returns:

The one-hot encoded NumPy array.

Return type:

np.ndarray

Example

>>> import edamame.eda as eda
>>> from sklearn import datasets
>>> iris = datasets.load_iris()
>>> y = iris.target
>>> y_ohe = eda.ohe(y)

edamame.eda.tools.scaling(X: DataFrame, minmaxscaler: bool = False) → DataFrame

The function returns the normalised/standardized matrix.

Parameters:

X (pd.DataFrame) – The model matrix X/X_train/X_test

minmaxscaler (bool): Select the type of scaling to apply to the numerical columns. By default is setted to the StandardScaler. If minmaxscaler is set to True the numerical columns is trasfomed to [0,1].

Returns:

pd.DataFrame

Example

>>> import edamame.eda as eda
>>> X = pd.DataFrame({'category': ['A', 'B', 'A', 'B', 'C', 'A'], 'value': [1, 2, 3, 4, 4, 5]})
>>> X = eda.scaling(X)

edamame.eda.tools.setup(X: DataFrame, y: DataFrame, dummy: bool = False, seed: int = 42, size: float = 0.25) → Tuple[DataFrame, DataFrame, DataFrame, DataFrame]

The function returns the following elements: X_train, y_train, X_test, y_test.

Parameters:

• X (pd.DataFrame) – The model matrix X (features matrix).

• y (pd.DataFrame) – The target variable.

• dummy (bool) – If False, the function produces the OHE. If True, the dummy encoding.

• seed (int) – Random seed to apply at the train_test_split function.

• size (float) – Size of the test dataset.

Returns:

X_train, y_train, X_test, y_test.

Return type:

Tuple[pd.DataFrame, pd.DataFrame, pd.DataFrame, pd.DataFrame]

Example

>>> import edamame.eda as eda
>>> df = pd.DataFrame({'category': ['A', 'B', 'A', 'B', 'C', 'A'], 'value': [1, 2, 3, 4, 4, 5], 'target': ['A2', 'A2', 'B2', 'B2', 'A2', 'B2']})
>>> X, y = eda.split_and_scaling(df, 'target')
>>> X_train, y_train, X_test, y_test = eda.setup(X, y)