Package 'timetk'

Description

The timetk package combines a collection of coercion tools for time series analysis.

Details

The timetk package has several benefits:

1. Visualizing Time Series

2. Wrangling Time Series.

3. Preprocessing and Feature Engineering.

To learn more about timetk, start with the documentation: https://business-science.github.io/timetk/

Author(s)

Maintainer: Matt Dancho [email protected]

Authors:

• Davis Vaughan [email protected]

See Also

Useful links:

• https://github.com/business-science/timetk

• https://business-science.github.io/timetk/

• Report bugs at https://github.com/business-science/timetk/issues

Description

anomalize() is used to detect anomalies in time series data, either for a single time series or for multiple time series grouped by a specific column.

Usage

anomalize(
  .data,
  .date_var,
  .value,
  .frequency = "auto",
  .trend = "auto",
  .method = "stl",
  .iqr_alpha = 0.05,
  .clean_alpha = 0.75,
  .max_anomalies = 0.2,
  .message = TRUE
)

Arguments

Details

The anomalize() method for anomaly detection that implements a 2-step process to detect outliers in time series.

Step 1: Detrend & Remove Seasonality using STL Decomposition

The decomposition separates the "season" and "trend" components from the "observed" values leaving the "remainder" for anomaly detection.

The user can control two parameters: frequency and trend.

1. .frequency: Adjusts the "season" component that is removed from the "observed" values.

2. .trend: Adjusts the trend window (t.window parameter from stats::stl() that is used.

The user may supply both .frequency and .trend as time-based durations (e.g. "6 weeks") or numeric values (e.g. 180) or "auto", which predetermines the frequency and/or trend based on the scale of the time series using the tk_time_scale_template().

Step 2: Anomaly Detection

Once "trend" and "season" (seasonality) is removed, anomaly detection is performed on the "remainder". Anomalies are identified, and boundaries (recomposed_l1 and recomposed_l2) are determined.

The Anomaly Detection Method uses an inner quartile range (IQR) of +/-25 the median.

IQR Adjustment, alpha parameter

With the default alpha = 0.05, the limits are established by expanding the 25/75 baseline by an IQR Factor of 3 (3X). The IQR Factor = 0.15 / alpha (hence 3X with alpha = 0.05):

• To increase the IQR Factor controlling the limits, decrease the alpha, which makes it more difficult to be an outlier.

• Increase alpha to make it easier to be an outlier.

• The IQR outlier detection method is used in forecast::tsoutliers().

• A similar outlier detection method is used by Twitter's AnomalyDetection package.

• Both Twitter and Forecast tsoutliers methods have been implemented in Business Science's anomalize package.

Value

A tibble or data.frame with the following columns:

• observed: original data

• seasonal: seasonal component

• seasadaj: seasonal adjusted

• trend: trend component

• remainder: residual component

• anomaly: Yes/No flag for outlier detection

• anomaly score: distance from centerline

• anomaly direction: -1, 0, 1 inidicator for direction of the anomaly

• recomposed_l1: lower level bound of recomposed time series

• recomposed_l2: upper level bound of recomposed time series

• observed_clean: original data with anomalies interpolated

References

1. CLEVELAND, R. B., CLEVELAND, W. S., MCRAE, J. E., AND TERPENNING, I. STL: A Seasonal-Trend Decomposition Procedure Based on Loess. Journal of Official Statistics, Vol. 6, No. 1 (1990), pp. 3-73.

2. Owen S. Vallis, Jordan Hochenbaum and Arun Kejariwal (2014). A Novel Technique for Long-Term Anomaly Detection in the Cloud. Twitter Inc.

Examples

library(dplyr)

walmart_sales_weekly %>%
    filter(id %in% c("1_1", "1_3")) %>%
    group_by(id) %>%
    anomalize(Date, Weekly_Sales)

Description

The easiest way to filter time series date or date-time vectors. Returns a logical vector indicating which date or date-time values are within a range. See filter_by_time() for the data.frame (tibble) implementation.

Usage

between_time(index, start_date = "start", end_date = "end")

Arguments

Details

Pure Time Series Filtering Flexibilty

The start_date and end_date parameters are designed with flexibility in mind.

Each side of the time_formula is specified as the character 'YYYY-MM-DD HH:MM:SS', but powerful shorthand is available. Some examples are:

• Year: ⁠start_date = '2013', end_date = '2015'⁠

• Month: ⁠start_date = '2013-01', end_date = '2016-06'⁠

• Day: ⁠start_date = '2013-01-05', end_date = '2016-06-04'⁠

• Second: ⁠start_date = '2013-01-05 10:22:15', end_date = '2018-06-03 12:14:22'⁠

• Variations: ⁠start_date = '2013', end_date = '2016-06'⁠

Key Words: "start" and "end"

Use the keywords "start" and "end" as shorthand, instead of specifying the actual start and end values. Here are some examples:

• Start of the series to end of 2015: ⁠start_date = 'start', end_date = '2015'⁠

• Start of 2014 to end of series: ⁠start_date = '2014', end_date = 'end'⁠

Internal Calculations

All shorthand dates are expanded:

• The start_date is expanded to be the first date in that period

• The end_date side is expanded to be the last date in that period

This means that the following examples are equivalent (assuming your index is a POSIXct):

• start_date = '2015' is equivalent to start_date = '2015-01-01 + 00:00:00'

• end_date = '2016' is equivalent to ⁠2016-12-31 + 23:59:59'⁠

Value

A logical vector the same length as index indicating whether or not the timestamp value was within the start_date and end_date range.

References

• This function is based on the tibbletime::filter_time() function developed by Davis Vaughan.

See Also

Time-Based dplyr functions:

• summarise_by_time() - Easily summarise using a date column.

• mutate_by_time() - Simplifies applying mutations by time windows.

• pad_by_time() - Insert time series rows with regularly spaced timestamps

• filter_by_time() - Quickly filter using date ranges.

• filter_period() - Apply filtering expressions inside periods (windows)

• slice_period() - Apply slice inside periods (windows)

• condense_period() - Convert to a different periodicity

• between_time() - Range detection for date or date-time sequences.

• slidify() - Turn any function into a sliding (rolling) function

Examples

library(dplyr)

index_daily <- tk_make_timeseries("2016-01-01", "2017-01-01", by = "day")
index_min   <- tk_make_timeseries("2016-01-01", "2017-01-01", by = "min")

# How it works
# - Returns TRUE/FALSE length of index
# - Use sum() to tally the number of TRUE values
index_daily %>% between_time("start", "2016-01") %>% sum()

# ---- INDEX SLICING ----

# Daily Series: Month of January 2016
index_daily[index_daily %>% between_time("start", "2016-01")]

# Daily Series: March 1st - June 15th, 2016
index_daily[index_daily %>% between_time("2016-03", "2016-06-15")]

# Minute Series:
index_min[index_min %>% between_time("2016-02-01 12:00", "2016-02-01 13:00")]

# ---- FILTERING WITH DPLYR ----
FANG %>%
    group_by(symbol) %>%
    filter(date %>% between_time("2016-01", "2016-01"))

Description

This dataset contains the daily count of rental bike transactions between years 2011 and 2012 in Capital bikeshare system with the corresponding weather and seasonal information.

Usage

bike_sharing_daily

Format

A tibble: 731 x 16

• instant: record index

• dteday : date

• season : season (1:winter, 2:spring, 3:summer, 4:fall)

• yr : year (0: 2011, 1:2012)

• mnth : month ( 1 to 12)

• hr : hour (0 to 23)

• holiday : weather day is holiday or not

• weekday : day of the week

• workingday : if day is neither weekend nor holiday is 1, otherwise is 0.

• weathersit :

  • 1: Clear, Few clouds, Partly cloudy, Partly cloudy

  • 2: Mist + Cloudy, Mist + Broken clouds, Mist + Few clouds, Mist

  • 3: Light Snow, Light Rain + Thunderstorm + Scattered clouds, Light Rain + Scattered clouds

  • 4: Heavy Rain + Ice Pallets + Thunderstorm + Mist, Snow + Fog

• temp : Normalized temperature in Celsius. The values are derived via (t-t_min)/(t_max-t_min), t_min=-8, t_max=+39 (only in hourly scale)

• atemp: Normalized feeling temperature in Celsius. The values are derived via (t-t_min)/(t_max-t_min), t_min=-16, t_max=+50 (only in hourly scale)

• hum: Normalized humidity. The values are divided to 100 (max)

• windspeed: Normalized wind speed. The values are divided to 67 (max)

• casual: count of casual users

• registered: count of registered users

• cnt: count of total rental bikes including both casual and registered

References

Fanaee-T, Hadi, and Gama, Joao, 'Event labeling combining ensemble detectors and background knowledge', Progress in Artificial Intelligence (2013): pp. 1-15, Springer Berlin Heidelberg.

Examples

bike_sharing_daily

Description

This is mainly a wrapper for the BoxCox transformation from the forecast R package. The box_cox_vec() function performs the transformation. box_cox_inv_vec() inverts the transformation. auto_lambda() helps in selecting the optimal lambda value.

Usage

box_cox_vec(x, lambda = "auto", silent = FALSE)

box_cox_inv_vec(x, lambda)

auto_lambda(
  x,
  method = c("guerrero", "loglik"),
  lambda_lower = -1,
  lambda_upper = 2
)

Arguments

Details

The Box Cox transformation is a power transformation that is commonly used to reduce variance of a time series.

Automatic Lambda Selection

If desired, the lambda argument can be selected using auto_lambda(), a wrapper for the Forecast R Package's forecast::BoxCox.lambda() function. Use either of 2 methods:

1. "guerrero" - Minimizes the non-seasonal variance

2. "loglik" - Maximizes the log-likelihood of a linear model fit to x

Value

Returns a numeric vector that has been transformed.

References

• Forecast R Package

• Forecasting: Principles & Practices: Transformations & Adjustments

• Guerrero, V.M. (1993) Time-series analysis supported by power transformations. Journal of Forecasting, 12, 37–48.

See Also

• Box Cox Transformation: box_cox_vec()

• Lag Transformation: lag_vec()

• Differencing Transformation: diff_vec()

• Rolling Window Transformation: slidify_vec()

• Loess Smoothing Transformation: smooth_vec()

• Fourier Series: fourier_vec()

• Missing Value Imputation for Time Series: ts_impute_vec(), ts_clean_vec()

Other common transformations to reduce variance: log(), log1p() and sqrt()

Examples

library(dplyr)
d10_daily <- m4_daily %>% dplyr::filter(id == "D10")

# --- VECTOR ----

value_bc <- box_cox_vec(d10_daily$value)
value    <- box_cox_inv_vec(value_bc, lambda = 1.25119350454964)

# --- MUTATE ----

m4_daily %>%
    dplyr::group_by(id) %>%
    dplyr::mutate(value_bc = box_cox_vec(value))

Description

Convert a data.frame object from daily to monthly, from minute data to hourly, and more. This allows the user to easily aggregate data to a less granular level by taking the value from either the beginning or end of the period.

Usage

condense_period(.data, .date_var, .period = "1 day", .side = c("start", "end"))

Arguments

Value

A tibble or data.frame

See Also

Time-Based dplyr functions:

• summarise_by_time() - Easily summarise using a date column.

• mutate_by_time() - Simplifies applying mutations by time windows.

• pad_by_time() - Insert time series rows with regularly spaced timestamps

• filter_by_time() - Quickly filter using date ranges.

• filter_period() - Apply filtering expressions inside periods (windows)

• slice_period() - Apply slice inside periods (windows)

• condense_period() - Convert to a different periodicity

• between_time() - Range detection for date or date-time sequences.

• slidify() - Turn any function into a sliding (rolling) function

Examples

# Libraries
library(dplyr)

# First value in each month
m4_daily %>%
    group_by(id) %>%
    condense_period(.period = "1 month")

# Last value in each month
m4_daily %>%
    group_by(id) %>%
    condense_period(.period = "1 month", .side = "end")

Description

diff_vec() applies a Differencing Transformation. diff_inv_vec() inverts the differencing transformation.

Usage

diff_vec(
  x,
  lag = 1,
  difference = 1,
  log = FALSE,
  initial_values = NULL,
  silent = FALSE
)

diff_inv_vec(x, lag = 1, difference = 1, log = FALSE, initial_values = NULL)

Arguments

Details

Benefits:

This function is NA padded by default so it works well with dplyr::mutate() operations.

Difference Calculation

Single differencing, diff_vec(x_t) is equivalent to: x_t - x_t1, where the subscript _t1 indicates the first lag. This transformation can be interpereted as change.

Double Differencing Calculation

Double differencing, diff_vec(x_t, difference = 2) is equivalent to: ⁠(x_t - x_t1) - (x_t - x_t1)_t1⁠, where the subscript _t1 indicates the first lag. This transformation can be interpereted as acceleration.

Log Difference Calculation

Log differencing, diff_vec(x_t, log = TRUE) is equivalent to: log(x_t) - log(x_t1) = log(x_t / x_t1), where x_t is the series and x_t1 is the first lag.

The 1st difference diff_vec(difference = 1, log = TRUE) has an interesting property where diff_vec(difference = 1, log = TRUE) %>% exp() is approximately 1 + rate of change.

Value

A numeric vector

See Also

Advanced Differencing and Modeling:

• step_diff() - Recipe for tidymodels workflow

• tk_augment_differences() - Adds many differences to a data.frame (tibble)

Additional Vector Functions:

• Box Cox Transformation: box_cox_vec()

• Lag Transformation: lag_vec()

• Differencing Transformation: diff_vec()

• Rolling Window Transformation: slidify_vec()

• Loess Smoothing Transformation: smooth_vec()

• Fourier Series: fourier_vec()

• Missing Value Imputation for Time Series: ts_impute_vec(), ts_clean_vec()

Examples

library(dplyr)

# --- USAGE ----

diff_vec(1:10, lag = 2, difference = 2) %>%
    diff_inv_vec(lag = 2, difference = 2, initial_values = 1:4)

# --- VECTOR ----

# Get Change
1:10 %>% diff_vec()

# Get Acceleration
1:10 %>% diff_vec(difference = 2)

# Get approximate rate of change
1:10 %>% diff_vec(log = TRUE) %>% exp() - 1


# --- MUTATE ----

m4_daily %>%
    group_by(id) %>%
    mutate(difference = diff_vec(value, lag = 1)) %>%
    mutate(
        difference_inv = diff_inv_vec(
            difference,
            lag = 1,
            # Add initial value to calculate the inverse difference
            initial_values = value[1]
        )
    )

Description

A dataset containing the daily historical stock prices for the "FANG" tech stocks, "FB", "AMZN", "NFLX", and "GOOG", spanning from the beginning of 2013 through the end of 2016.

Usage

FANG

Format

A "tibble" ("tidy" data frame) with 4,032 rows and 8 variables:

symbol

stock ticker symbol

date

trade date

open

stock price at the open of trading, in USD

high

stock price at the highest point during trading, in USD

low

stock price at the lowest point during trading, in USD

close

stock price at the close of trading, in USD

volume

number of shares traded

adjusted

stock price at the close of trading adjusted for stock splits, in USD

Description

The easiest way to filter time-based start/end ranges using shorthand timeseries notation. See filter_period() for applying filter expression by period (windows).

Usage

filter_by_time(.data, .date_var, .start_date = "start", .end_date = "end")

Arguments

Details

Pure Time Series Filtering Flexibilty

The .start_date and .end_date parameters are designed with flexibility in mind.

Each side of the time_formula is specified as the character 'YYYY-MM-DD HH:MM:SS', but powerful shorthand is available. Some examples are:

• Year: ⁠.start_date = '2013', .end_date = '2015'⁠

• Month: ⁠.start_date = '2013-01', .end_date = '2016-06'⁠

• Day: ⁠.start_date = '2013-01-05', .end_date = '2016-06-04'⁠

• Second: ⁠.start_date = '2013-01-05 10:22:15', .end_date = '2018-06-03 12:14:22'⁠

• Variations: ⁠.start_date = '2013', .end_date = '2016-06'⁠

Key Words: "start" and "end"

Use the keywords "start" and "end" as shorthand, instead of specifying the actual start and end values. Here are some examples:

• Start of the series to end of 2015: ⁠.start_date = 'start', .end_date = '2015'⁠

• Start of 2014 to end of series: ⁠.start_date = '2014', .end_date = 'end'⁠

Internal Calculations

All shorthand dates are expanded:

• The .start_date is expanded to be the first date in that period

• The .end_date side is expanded to be the last date in that period

This means that the following examples are equivalent (assuming your index is a POSIXct):

• .start_date = '2015' is equivalent to .start_date = '2015-01-01 + 00:00:00'

• .end_date = '2016' is equivalent to ⁠2016-12-31 + 23:59:59'⁠

Value

Returns a tibble or data.frame that has been filtered.

References

• This function is based on the tibbletime::filter_time() function developed by Davis Vaughan.

See Also

Time-Based dplyr functions:

• summarise_by_time() - Easily summarise using a date column.

• mutate_by_time() - Simplifies applying mutations by time windows.

• pad_by_time() - Insert time series rows with regularly spaced timestamps

• filter_by_time() - Quickly filter using date ranges.

• filter_period() - Apply filtering expressions inside periods (windows)

• slice_period() - Apply slice inside periods (windows)

• condense_period() - Convert to a different periodicity

• between_time() - Range detection for date or date-time sequences.

• slidify() - Turn any function into a sliding (rolling) function

Examples

library(dplyr)

# Filter values in January 1st through end of February, 2013
FANG %>%
    group_by(symbol) %>%
    filter_by_time(.start_date = "start", .end_date = "2013-02") %>%
    plot_time_series(date, adjusted, .facet_ncol = 2, .interactive = FALSE)

Description

Applies a dplyr filtering expression inside a time-based period (window). See filter_by_time() for filtering continuous ranges defined by start/end dates. filter_period() enables filtering expressions like:

• Filtering to the maximum value each month.

• Filtering the first date each month.

• Filtering all rows with value greater than a monthly average

Usage

filter_period(.data, ..., .date_var, .period = "1 day")

Arguments

Value

A tibble or data.frame

See Also

Time-Based dplyr functions:

• summarise_by_time() - Easily summarise using a date column.

• mutate_by_time() - Simplifies applying mutations by time windows.

• pad_by_time() - Insert time series rows with regularly spaced timestamps

• filter_by_time() - Quickly filter using date ranges.

• filter_period() - Apply filtering expressions inside periods (windows)

• slice_period() - Apply slice inside periods (windows)

• condense_period() - Convert to a different periodicity

• between_time() - Range detection for date or date-time sequences.

• slidify() - Turn any function into a sliding (rolling) function

Examples

# Libraries
library(dplyr)

# Max value in each month
m4_daily %>%
    group_by(id) %>%
    filter_period(.period = "1 month", value == max(value))

# First date each month
m4_daily %>%
    group_by(id) %>%
    filter_period(.period = "1 month", date == first(date))

# All observations that are greater than a monthly average
m4_daily %>%
    group_by(id) %>%
    filter_period(.period = "1 month", value > mean(value))

Description

fourier_vec() calculates a Fourier Series from a date or date-time index.

Usage

fourier_vec(x, period, K = 1, type = c("sin", "cos"), scale_factor = NULL)

Arguments

Details

Benefits:

This function is NA padded by default so it works well with dplyr::mutate() operations.

Fourier Series Calculation

The internal calculation is relatively straightforward: ⁠fourier(x) = sin(2 * pi * term * x) or cos(2 * pi * term * x)⁠, where term = K / period.

Period Alignment, period

The period alignment with the sequence is an essential part of fourier series calculation.

• Date, Date-Time, and Zoo (yearqtr and yearmon) Sequences - Are scaled to unit difference of 1. This happens internally, so there's nothing you need to do or to worry about. Future time series will be scaled appropriately.

• Numeric Sequences - Are not scaled, which means you should transform them to a unit difference of 1 so that your x is a sequence that increases by 1. Otherwise your period and fourier order will be incorrectly calculated. The solution is to just take your sequence and divide by the median difference between values.

Fourier Order, K

The fourier order is a parameter that increases the frequency. K = 2 doubles the frequency. It's common in time series analysis to add multiple fourier orders (e.g. 1 through 5) to account for seasonalities that occur faster than the primary seasonality.

Type (Sin/Cos)

The type of the fourier series can be either sin or cos. It's common in time series analysis to add both sin and cos series.

Value

A numeric vector

See Also

Fourier Modeling Functions:

• step_fourier() - Recipe for tidymodels workflow

• tk_augment_fourier() - Adds many fourier series to a data.frame (tibble)

Additional Vector Functions:

• Fourier Series: fourier_vec()

• Box Cox Transformation: box_cox_vec()

• Lag Transformation: lag_vec()

• Differencing Transformation: diff_vec()

• Rolling Window Transformation: slidify_vec()

• Loess Smoothing Transformation: smooth_vec()

• Missing Value Imputation for Time Series: ts_impute_vec(), ts_clean_vec()

Examples

library(dplyr)

# Set max.print to 50
options_old <- options()$max.print
options(max.print = 50)

date_sequence <- tk_make_timeseries("2016-01-01", "2016-01-31", by = "hour")

# --- VECTOR ---

fourier_vec(date_sequence, period = 7 * 24, K = 1, type = "sin")

# --- MUTATE ---

tibble(date = date_sequence) %>%
    # Add cosine series that oscilates at a 7-day period
    mutate(
        C1_7 = fourier_vec(date, period = 7*24, K = 1, type = "cos"),
        C2_7 = fourier_vec(date, period = 7*24, K = 2, type = "cos")
    ) %>%
    # Visualize
    tidyr::pivot_longer(cols = contains("_"), names_to = "name", values_to = "value") %>%
    plot_time_series(
        date, value, .color_var = name,
        .smooth = FALSE,
        .interactive = FALSE,
        .title = "7-Day Fourier Terms"
    )

options(max.print = options_old)

Description

Make future time series from existing

Usage

future_frame(
  .data,
  .date_var,
  .length_out,
  .inspect_weekdays = FALSE,
  .inspect_months = FALSE,
  .skip_values = NULL,
  .insert_values = NULL,
  .bind_data = FALSE
)

Arguments

Details

This is a wrapper for tk_make_future_timeseries() that works on data.frames. It respects dplyr groups.

Specifying Length of Future Observations

The argument .length_out determines how many future index observations to compute. It can be specified as:

• A numeric value - the number of future observations to return.

  • The number of observations returned is always equal to the value the user inputs.

  • The end date can vary based on the number of timestamps chosen.

• A time-based phrase - The duration into the future to include (e.g. "6 months" or "30 minutes").

  • The duration defines the end date for observations.

  • The end date will not change and those timestamps that fall within the end date will be returned (e.g. a quarterly time series will return 4 quarters if .length_out = "1 year").

  • The number of observations will vary to fit within the end date.

Weekday and Month Inspection

The .inspect_weekdays and .inspect_months arguments apply to "daily" (scale = "day") data (refer to tk_get_timeseries_summary() to get the index scale).

• The .inspect_weekdays argument is useful in determining missing days of the week that occur on a weekly frequency such as every week, every other week, and so on. It's recommended to have at least 60 days to use this option.

• The .inspect_months argument is useful in determining missing days of the month, quarter or year; however, the algorithm can inadvertently select incorrect dates if the pattern is erratic.

Skipping / Inserting Values

The .skip_values and .insert_values arguments can be used to remove and add values into the series of future times. The values must be the same format as the idx class.

• The .skip_values argument useful for passing holidays or special index values that should be excluded from the future time series.

• The .insert_values argument is useful for adding values back that the algorithm may have excluded.

Binding with Data

Rowwise binding with the original is so common that I've added an argument .bind_data to perform a row-wise bind of the future data and the incoming data.

This replaces the need to do:

df %>%
   future_frame(.length_out = "6 months") %>%
   bind_rows(df, .)

Now you can just do:

df %>%
    future_frame(.length_out = "6 months", .bind_data = TRUE)

Value

A tibble that has been extended with future date, date-time timestamps.

See Also

• Making Future Time Series: tk_make_future_timeseries() (Underlying function)

Examples

library(dplyr)

# 30-min interval data
taylor_30_min %>%
    future_frame(date, .length_out = "1 week")

# Daily Data (Grouped)
m4_daily %>%
    group_by(id) %>%
    future_frame(date, .length_out = "6 weeks")

# Specify how many observations to project into the future
m4_daily %>%
    group_by(id) %>%
    future_frame(date, .length_out = 100)

# Bind with Original Data
m4_daily %>%
    group_by(id) %>%
    future_frame(date, .length_out = 100, .bind_data = TRUE)

holidays <- tk_make_holiday_sequence(
    start_date = "2017-01-01",
    end_date   = "2017-12-31",
    calendar   = "NYSE")

weekends <- tk_make_weekend_sequence(
    start_date = "2017-01-01",
    end_date   = "2017-12-31"
)

FANG %>%
    group_by(symbol) %>%
    future_frame(
        .length_out       = "1 year",
        .skip_values      = c(holidays, weekends)
    )

Description

Check if an object is a date class

Usage

is_date_class(x)

Arguments

Value

Logical (TRUE/FALSE)

Examples

library(dplyr)

tk_make_timeseries("2011") %>% is_date_class()

letters %>% is_date_class()

Description

lag_vec() applies a Lag Transformation.

Usage

lag_vec(x, lag = 1)

lead_vec(x, lag = -1)

Arguments

Details

Benefits:

This function is NA padded by default so it works well with dplyr::mutate() operations. The function allows both lags and leads (negative lags).

Lag Calculation

A lag is an offset of lag periods. NA values are returned for the number of lag periods.

Lead Calculation

A negative lag is considered a lead. The only difference between lead_vec() and lag_vec() is that the lead_vec() function contains a starting negative value.

Value

A numeric vector

See Also

Modeling and Advanced Lagging:

• recipes::step_lag() - Recipe for adding lags in tidymodels modeling

• tk_augment_lags() - Add many lags group-wise to a data.frame (tibble)

Vectorized Transformations:

• Box Cox Transformation: box_cox_vec()

• Lag Transformation: lag_vec()

• Differencing Transformation: diff_vec()

• Rolling Window Transformation: slidify_vec()

• Loess Smoothing Transformation: smooth_vec()

• Fourier Series: fourier_vec()

• Missing Value Imputation for Time Series: ts_impute_vec(), ts_clean_vec()

Examples

library(dplyr)

# --- VECTOR ----

# Lag
1:10 %>% lag_vec(lag = 1)

# Lead
1:10 %>% lag_vec(lag = -1)


# --- MUTATE ----

m4_daily %>%
    group_by(id) %>%
    mutate(lag_1 = lag_vec(value, lag = 1))

Description

The log_interval_vec() transformation constrains a forecast to an interval specified by an upper_limit and a lower_limit. The transformation provides similar benefits to log() transformation, while ensuring the inverted transformation stays within an upper and lower limit.

Usage

log_interval_vec(
  x,
  limit_lower = "auto",
  limit_upper = "auto",
  offset = 0,
  silent = FALSE
)

log_interval_inv_vec(x, limit_lower, limit_upper, offset = 0)

Arguments

Details

Log Interval Transformation

The Log Interval Transformation constrains values to specified upper and lower limits. The transformation maps limits to a function:

log(((x + offset) - a)/(b - (x + offset)))

where a is the lower limit and b is the upper limit

Inverse Transformation

The inverse transformation:

(b-a)*(exp(x)) / (1 + exp(x)) + a - offset

Value

A numeric vector of the transformed series.

References

• Forecasting: Principles & Practices: Forecasts constrained to an interval

See Also

• Box Cox Transformation: box_cox_vec()

• Lag Transformation: lag_vec()

• Differencing Transformation: diff_vec()

• Rolling Window Transformation: slidify_vec()

• Loess Smoothing Transformation: smooth_vec()

• Fourier Series: fourier_vec()

• Missing Value Imputation & Anomaly Cleaning for Time Series: ts_impute_vec(), ts_clean_vec()

Other common transformations to reduce variance: log(), log1p() and sqrt()

Examples

library(dplyr)

values_trans <- log_interval_vec(1:10, limit_lower = 0, limit_upper = 11)
values_trans

values_trans_forecast <- c(values_trans, 3.4, 4.4, 5.4)

values_trans_forecast %>%
    log_interval_inv_vec(limit_lower = 0, limit_upper = 11) %>%
    plot()

Description

The fourth M Competition. M4, started on 1 January 2018 and ended in 31 May 2018. The competition included 100,000 time series datasets. This dataset includes a sample of 4 daily time series from the competition.

Usage

m4_daily

Format

A tibble: 9,743 x 3

• id Factor. Unique series identifier (4 total)

• date Date. Timestamp information. Daily format.

• value Numeric. Value at the corresponding timestamp.

Details

This is a sample of 4 daily data sets from the M4 competition.

Source

• M4 Competition Website

Examples

m4_daily

Description

The fourth M Competition. M4, started on 1 January 2018 and ended in 31 May 2018. The competition included 100,000 time series datasets. This dataset includes a sample of 4 hourly time series from the competition.

Usage

m4_hourly

Format

A tibble: 3,060 x 3

• id Factor. Unique series identifier (4 total)

• date Date-time. Timestamp information. Hourly format.

• value Numeric. Value at the corresponding timestamp.

Details

This is a sample of 4 hourly data sets from the M4 competition.

Source

• M4 Competition Website

Examples

m4_hourly

Description

The fourth M Competition. M4, started on 1 January 2018 and ended in 31 May 2018. The competition included 100,000 time series datasets. This dataset includes a sample of 4 monthly time series from the competition.

Usage

m4_monthly

Format

A tibble: 9,743 x 3

• id Factor. Unique series identifier (4 total)

• date Date. Timestamp information. Monthly format.

• value Numeric. Value at the corresponding timestamp.

Details

This is a sample of 4 Monthly data sets from the M4 competition.

Source

• M4 Competition Website

Examples

m4_monthly

Description

The fourth M Competition. M4, started on 1 January 2018 and ended in 31 May 2018. The competition included 100,000 time series datasets. This dataset includes a sample of 4 quarterly time series from the competition.

Usage

m4_quarterly

Format

A tibble: 9,743 x 3

• id Factor. Unique series identifier (4 total)

• date Date. Timestamp information. Quarterly format.

• value Numeric. Value at the corresponding timestamp.

Details

This is a sample of 4 Quarterly data sets from the M4 competition.

Source

• M4 Competition Website

Examples

m4_quarterly

Description

The fourth M Competition. M4, started on 1 January 2018 and ended in 31 May 2018. The competition included 100,000 time series datasets. This dataset includes a sample of 4 weekly time series from the competition.

Usage

m4_weekly

Format

A tibble: 9,743 x 3

• id Factor. Unique series identifier (4 total)

• date Date. Timestamp information. Weekly format.

• value Numeric. Value at the corresponding timestamp.

Details

This is a sample of 4 Weekly data sets from the M4 competition.

Source

• M4 Competition Website

Examples

m4_weekly

Description

The fourth M Competition. M4, started on 1 January 2018 and ended in 31 May 2018. The competition included 100,000 time series datasets. This dataset includes a sample of 4 yearly time series from the competition.

Usage

m4_yearly

Format

A tibble: 9,743 x 3

• id Factor. Unique series identifier (4 total)

• date Date. Timestamp information. Yearly format.

• value Numeric. Value at the corresponding timestamp.

Details

This is a sample of 4 Yearly data sets from the M4 competition.

Source

• M4 Competition Website

Examples

m4_yearly

Description

mutate_by_time() is a time-based variant of the popular dplyr::mutate() function that uses .date_var to specify a date or date-time column and .by to group the calculation by groups like "5 seconds", "week", or "3 months".

Usage

mutate_by_time(
  .data,
  .date_var,
  .by = "day",
  ...,
  .type = c("floor", "ceiling", "round")
)

Arguments

Value

A tibble or data.frame

See Also

Time-Based dplyr functions:

• summarise_by_time() - Easily summarise using a date column.

• mutate_by_time() - Simplifies applying mutations by time windows.

• pad_by_time() - Insert time series rows with regularly spaced timestamps

• filter_by_time() - Quickly filter using date ranges.

• filter_period() - Apply filtering expressions inside periods (windows)

• slice_period() - Apply slice inside periods (windows)

• condense_period() - Convert to a different periodicity

• between_time() - Range detection for date or date-time sequences.

• slidify() - Turn any function into a sliding (rolling) function

Examples

# Libraries
library(dplyr)

# First value in each month
m4_daily_first_by_month_tbl <- m4_daily %>%
    group_by(id) %>%
    mutate_by_time(
        .date_var = date,
        .by       = "month", # Setup for monthly aggregation
        # mutate recycles a single value
        first_value_by_month  = first(value)
    )
m4_daily_first_by_month_tbl

# Visualize Time Series vs 1st Value Each Month
m4_daily_first_by_month_tbl %>%
    tidyr::pivot_longer(value:first_value_by_month) %>%
    plot_time_series(date, value, name,
                     .facet_scale = "free", .facet_ncol = 2,
                     .smooth = FALSE, .interactive = FALSE)

Description

Normalization is commonly used to center and scale numeric features to prevent one from dominating in algorithms that require data to be on the same scale.

Usage

normalize_vec(x, min = NULL, max = NULL, silent = FALSE)

normalize_inv_vec(x, min, max)

Arguments

Details

Standardization vs Normalization

• Standardization refers to a transformation that reduces the range to mean 0, standard deviation 1

• Normalization refers to a transformation that reduces the min-max range: (0, 1)

Value

A numeric vector with the transformation applied.

See Also

• Normalization/Standardization: standardize_vec(), normalize_vec()

• Box Cox Transformation: box_cox_vec()

• Lag Transformation: lag_vec()

• Differencing Transformation: diff_vec()

• Rolling Window Transformation: slidify_vec()

• Loess Smoothing Transformation: smooth_vec()

• Fourier Series: fourier_vec()

• Missing Value Imputation for Time Series: ts_impute_vec(), ts_clean_vec()

Examples

library(dplyr)

d10_daily <- m4_daily %>% dplyr::filter(id == "D10")

# --- VECTOR ----

value_norm <- normalize_vec(d10_daily$value)
value      <- normalize_inv_vec(value_norm,
                                min = 1781.6,
                                max = 2649.3)

# --- MUTATE ----

m4_daily %>%
    group_by(id) %>%
    mutate(value_norm = normalize_vec(value))

Description

The easiest way to fill in missing timestamps or convert to a more granular period (e.g. quarter to month). Wraps the padr::pad() function for padding tibbles.

Usage

pad_by_time(
  .data,
  .date_var,
  .by = "auto",
  .pad_value = NA,
  .fill_na_direction = c("none", "down", "up", "downup", "updown"),
  .start_date = NULL,
  .end_date = NULL
)

Arguments

Details

Padding Missing Observations

The most common use case for pad_by_time() is to add rows where timestamps are missing. This could be from sales data that have missing values on weekends and holidays. Or it could be high frequency data where observations are irregularly spaced and should be reset to a regular frequency.

Going from Low to High Frequency

The second use case is going from a low frequency (e.g. day) to high frequency (e.g. hour). This is possible by supplying a higher frequency to pad_by_time().

Interval, .by

Padding can be applied in the following ways:

• .by = "auto" - pad_by_time() will detect the time-stamp frequency and apply padding.

• The eight intervals in are: year, quarter, month, week, day, hour, min, and sec.

• Intervals like 5 minutes, 6 hours, 10 days are possible.

Pad Value, .pad_value

A pad value can be supplied that fills in missing numeric data. Note that this is only applied to numeric columns.

Fill NA Direction, .fill_na_directions

Uses tidyr::fill() to fill missing observations using a fill strategy.

Value

A tibble or data.frame with rows containing missing timestamps added.

References

• This function wraps the padr::pad() function developed by Edwin Thoen.

See Also

Imputation:

• ts_impute_vec() - Impute missing values for time series.

Time-Based dplyr functions:

• summarise_by_time() - Easily summarise using a date column.

• mutate_by_time() - Simplifies applying mutations by time windows.

• pad_by_time() - Insert time series rows with regularly spaced timestamps

• filter_by_time() - Quickly filter using date ranges.

• filter_period() - Apply filtering expressions inside periods (windows)

• slice_period() - Apply slice inside periods (windows)

• condense_period() - Convert to a different periodicity

• between_time() - Range detection for date or date-time sequences.

• slidify() - Turn any function into a sliding (rolling) function

Examples

library(dplyr)

# Create a quarterly series with 1 missing value
missing_data_tbl <- tibble::tibble(
    date = tk_make_timeseries("2014-01-01", "2015-01-01", by = "quarter"),
    value = 1:5
) %>%
    slice(-4) # Lose the 4th quarter on purpose
missing_data_tbl


# Detects missing quarter, and pads the missing regularly spaced quarter with NA
missing_data_tbl %>% pad_by_time(date, .by = "quarter")

# Can specify a shorter period. This fills monthly.
missing_data_tbl %>% pad_by_time(date, .by = "month")

# Can let pad_by_time() auto-detect date and period
missing_data_tbl %>% pad_by_time()

# Can specify a .pad_value
missing_data_tbl %>% pad_by_time(date, .by = "quarter", .pad_value = 0)

# Can then impute missing values
missing_data_tbl %>%
    pad_by_time(date, .by = "quarter") %>%
    mutate(value = ts_impute_vec(value, period = 1))

# Can specify a custom .start_date and .end_date
missing_data_tbl %>%
   pad_by_time(date, .by = "quarter", .start_date = "2013", .end_date = "2015-07-01")

# Can specify a tidyr::fill() direction
missing_data_tbl %>%
   pad_by_time(date, .by = "quarter",
               .fill_na_direction = "downup",
               .start_date = "2013", .end_date = "2015-07-01")

# --- GROUPS ----

# Apply standard NA padding to groups
FANG %>%
    group_by(symbol) %>%
    pad_by_time(.by = "day")

# Apply constant pad value
FANG %>%
    group_by(symbol) %>%
    pad_by_time(.by = "day", .pad_value = 0)

# Apply filled padding to groups
FANG %>%
    group_by(symbol) %>%
    pad_by_time(.by = "day", .fill_na_direction = "down")

Description

Significantly faster time series parsing than readr::parse_date, readr::parse_datetime, lubridate::as_date(), and lubridate::as_datetime(). Uses anytime package, which relies on Boost.Date_Time C++ library for date/datetime parsing.

Usage

parse_date2(x, ..., silent = FALSE)

parse_datetime2(x, tz = "UTC", tz_shift = FALSE, ..., silent = FALSE)

Arguments

Details

Parsing Formats

• Date Formats: Must follow a Year, Month, Day sequence. (e.g. parse_date2("2011 June") is OK, parse_date2("June 2011") is NOT OK).

• Date Time Formats: Must follow a YMD HMS sequence.

Refer to lubridate::mdy() for Month, Day, Year and additional formats.

Time zones (Datetime)

Time zones are handled in a similar way to lubridate::as_datetime() in that time zones are forced rather than shifted. This is a key difference between anytime::anytime(), which shifts datetimes to the specified timezone by default.

Value

Returns a date or datatime vector from the transformation applied to character timestamp vector.

References

• This function wraps the anytime::anytime() and anytime::anydate() functions developed by Dirk Eddelbuettel.

Examples

# Fast date parsing
parse_date2("2011")
parse_date2("2011 June 3rd")

# Fast datetime parsing
parse_datetime2("2011")
parse_datetime2("2011 Jan 1 12:35:21")

# Time Zones (datetime only)
parse_datetime2("2011 Jan 1 12:35:21", tz = "Europe/London")

Description

Returns the ACF and PACF of a target and optionally CCF's of one or more lagged predictors in interactive plotly plots. Scales to multiple time series with group_by().

Usage

plot_acf_diagnostics(
  .data,
  .date_var,
  .value,
  .ccf_vars = NULL,
  .lags = 1000,
  .show_ccf_vars_only = FALSE,
  .show_white_noise_bars = TRUE,
  .facet_ncol = 1,
  .facet_scales = "fixed",
  .line_color = "#2c3e50",
  .line_size = 0.5,
  .line_alpha = 1,
  .point_color = "#2c3e50",
  .point_size = 1,
  .point_alpha = 1,
  .x_intercept = NULL,
  .x_intercept_color = "#E31A1C",
  .hline_color = "#2c3e50",
  .white_noise_line_type = 2,
  .white_noise_line_color = "#A6CEE3",
  .title = "Lag Diagnostics",
  .x_lab = "Lag",
  .y_lab = "Correlation",
  .interactive = TRUE,
  .plotly_slider = FALSE
)

Arguments

Details

Simplified ACF, PACF, & CCF

We are often interested in all 3 of these functions. Why not get all 3+ at once? Now you can.

• ACF - Autocorrelation between a target variable and lagged versions of itself

• PACF - Partial Autocorrelation removes the dependence of lags on other lags highlighting key seasonalities.

• CCF - Shows how lagged predictors can be used for prediction of a target variable.

Lag Specification

Lags (.lags) can either be specified as:

• A time-based phrase indicating a duraction (e.g. ⁠2 months⁠)

• A maximum lag (e.g. .lags = 28)

• A sequence of lags (e.g. .lags = 7:28)

Scales to Multiple Time Series with Groups

The plot_acf_diagnostics() works with grouped_df's, meaning you can group your time series by one or more categorical columns with dplyr::group_by() and then apply plot_acf_diagnostics() to return group-wise lag diagnostics.

Special Note on Groups

Unlike other plotting utilities, the .facet_vars arguments is NOT included. Use dplyr::group_by() for processing multiple time series groups.

Calculating the White Noise Significance Bars

The formula for the significance bars is +2/sqrt(T) and -2/sqrt(T) where T is the length of the time series. For a white noise time series, 95% of the data points should fall within this range. Those that don't may be significant autocorrelations.

Value

A static ggplot2 plot or an interactive plotly plot

See Also

• Visualizing ACF, PACF, & CCF: plot_acf_diagnostics()

• Visualizing Seasonality: plot_seasonal_diagnostics()

• Visualizing Time Series: plot_time_series()

Examples

library(dplyr)
library(ggplot2)


# Apply Transformations
# - Differencing transformation to identify ARIMA & SARIMA Orders
m4_hourly %>%
    group_by(id) %>%
    plot_acf_diagnostics(
        date, value,               # ACF & PACF
        .lags = "7 days",          # 7-Days of hourly lags
        .interactive = FALSE
    )

# Apply Transformations
# - Differencing transformation to identify ARIMA & SARIMA Orders
m4_hourly %>%
    group_by(id) %>%
    plot_acf_diagnostics(
        date,
        diff_vec(value, lag = 1), # Difference the value column
        .lags        = 0:(24*7),   # 7-Days of hourly lags
        .interactive = FALSE
    ) +
    ggtitle("ACF Diagnostics",  subtitle = "1st Difference")

# CCFs Too!
walmart_sales_weekly %>%
    select(id, Date, Weekly_Sales, Temperature, Fuel_Price) %>%
    group_by(id) %>%
    plot_acf_diagnostics(
        Date, Weekly_Sales,                        # ACF & PACF
        .ccf_vars    = c(Temperature, Fuel_Price), # CCFs
        .lags        = "3 months", # 3 months of weekly lags
        .interactive = FALSE
    )

Description

plot_anomalies() is an interactive and scalable function for visualizing anomalies in time series data. Plots are available in interactive plotly (default) and static ggplot2 format.

plot_anomalies_decomp(): Takes in data from the anomalize() function, and returns a plot of the anomaly decomposition. Useful for interpeting how the anomalize() function is determining outliers from "remainder".

plot_anomalies_cleaned() helps users visualize the before/after of cleaning anomalies.

Usage

plot_anomalies(
  .data,
  .date_var,
  .facet_vars = NULL,
  .facet_ncol = 1,
  .facet_nrow = 1,
  .facet_scales = "free",
  .facet_dir = "h",
  .facet_collapse = FALSE,
  .facet_collapse_sep = " ",
  .facet_strip_remove = FALSE,
  .line_color = "#2c3e50",
  .line_size = 0.5,
  .line_type = 1,
  .line_alpha = 1,
  .anom_color = "#e31a1c",
  .anom_alpha = 1,
  .anom_size = 1.5,
  .ribbon_fill = "grey20",
  .ribbon_alpha = 0.2,
  .legend_show = TRUE,
  .title = "Anomaly Plot",
  .x_lab = "",
  .y_lab = "",
  .color_lab = "Anomaly",
  .interactive = TRUE,
  .trelliscope = FALSE,
  .trelliscope_params = list()
)

plot_anomalies_decomp(
  .data,
  .date_var,
  .facet_vars = NULL,
  .facet_scales = "free",
  .line_color = "#2c3e50",
  .line_size = 0.5,
  .line_type = 1,
  .line_alpha = 1,
  .title = "Anomaly Decomposition Plot",
  .x_lab = "",
  .y_lab = "",
  .interactive = TRUE
)

plot_anomalies_cleaned(
  .data,
  .date_var,
  .facet_vars = NULL,
  .facet_ncol = 1,
  .facet_nrow = 1,
  .facet_scales = "free",
  .facet_dir = "h",
  .facet_collapse = FALSE,
  .facet_collapse_sep = " ",
  .facet_strip_remove = FALSE,
  .line_color = "#2c3e50",
  .line_size = 0.5,
  .line_type = 1,
  .line_alpha = 1,
  .cleaned_line_color = "#e31a1c",
  .cleaned_line_size = 0.5,
  .cleaned_line_type = 1,
  .cleaned_line_alpha = 1,
  .legend_show = TRUE,
  .title = "Anomalies Cleaned Plot",
  .x_lab = "",
  .y_lab = "",
  .color_lab = "Legend",
  .interactive = TRUE,
  .trelliscope = FALSE,
  .trelliscope_params = list()
)

Arguments

Value

A plotly or ggplot2 visualization

Examples

# Plot Anomalies
library(dplyr)

walmart_sales_weekly %>%
    filter(id %in% c("1_1", "1_3")) %>%
    group_by(id) %>%
    anomalize(Date, Weekly_Sales) %>%
    plot_anomalies(Date, .facet_ncol = 2, .ribbon_alpha = 0.25, .interactive = FALSE)

# Plot Anomalies Decomposition
library(dplyr)

walmart_sales_weekly %>%
    filter(id %in% c("1_1", "1_3")) %>%
    group_by(id) %>%
    anomalize(Date, Weekly_Sales, .message = FALSE) %>%
    plot_anomalies_decomp(Date, .interactive = FALSE)

# Plot Anomalies Cleaned
library(dplyr)

walmart_sales_weekly %>%
    filter(id %in% c("1_1", "1_3")) %>%
    group_by(id) %>%
    anomalize(Date, Weekly_Sales, .message = FALSE) %>%
    plot_anomalies_cleaned(Date, .facet_ncol = 2, .interactive = FALSE)

Description

An interactive and scalable function for visualizing anomalies in time series data. Plots are available in interactive plotly (default) and static ggplot2 format.

Usage

plot_anomaly_diagnostics(
  .data,
  .date_var,
  .value,
  .facet_vars = NULL,
  .frequency = "auto",
  .trend = "auto",
  .alpha = 0.05,
  .max_anomalies = 0.2,
  .message = TRUE,
  .facet_ncol = 1,
  .facet_nrow = 1,
  .facet_scales = "free",
  .facet_dir = "h",
  .facet_collapse = FALSE,
  .facet_collapse_sep = " ",
  .facet_strip_remove = FALSE,
  .line_color = "#2c3e50",
  .line_size = 0.5,
  .line_type = 1,
  .line_alpha = 1,
  .anom_color = "#e31a1c",
  .anom_alpha = 1,
  .anom_size = 1.5,
  .ribbon_fill = "grey20",
  .ribbon_alpha = 0.2,
  .legend_show = TRUE,
  .title = "Anomaly Diagnostics",
  .x_lab = "",
  .y_lab = "",
  .color_lab = "Anomaly",
  .interactive = TRUE,
  .trelliscope = FALSE,
  .trelliscope_params = list()
)

Arguments

Details

The plot_anomaly_diagnostics() is a visualization wrapper for tk_anomaly_diagnostics() group-wise anomaly detection, implements a 2-step process to detect outliers in time series.

Step 1: Detrend & Remove Seasonality using STL Decomposition

The decomposition separates the "season" and "trend" components from the "observed" values leaving the "remainder" for anomaly detection.

The user can control two parameters: frequency and trend.

1. .frequency: Adjusts the "season" component that is removed from the "observed" values.

2. .trend: Adjusts the trend window (t.window parameter from stats::stl() that is used.

The user may supply both .frequency and .trend as time-based durations (e.g. "6 weeks") or numeric values (e.g. 180) or "auto", which predetermines the frequency and/or trend based on the scale of the time series using the tk_time_scale_template().

Step 2: Anomaly Detection

Once "trend" and "season" (seasonality) is removed, anomaly detection is performed on the "remainder". Anomalies are identified, and boundaries (recomposed_l1 and recomposed_l2) are determined.

The Anomaly Detection Method uses an inner quartile range (IQR) of +/-25 the median.

IQR Adjustment, alpha parameter

With the default alpha = 0.05, the limits are established by expanding the 25/75 baseline by an IQR Factor of 3 (3X). The IQR Factor = 0.15 / alpha (hence 3X with alpha = 0.05):

• To increase the IQR Factor controlling the limits, decrease the alpha, which makes it more difficult to be an outlier.

• Increase alpha to make it easier to be an outlier.

• The IQR outlier detection method is used in forecast::tsoutliers().

• A similar outlier detection method is used by Twitter's AnomalyDetection package.

• Both Twitter and Forecast tsoutliers methods have been implemented in Business Science's anomalize package.

Value

A plotly or ggplot2 visualization

References

1. CLEVELAND, R. B., CLEVELAND, W. S., MCRAE, J. E., AND TERPENNING, I. STL: A Seasonal-Trend Decomposition Procedure Based on Loess. Journal of Official Statistics, Vol. 6, No. 1 (1990), pp. 3-73.

2. Owen S. Vallis, Jordan Hochenbaum and Arun Kejariwal (2014). A Novel Technique for Long-Term Anomaly Detection in the Cloud. Twitter Inc.

See Also

• tk_anomaly_diagnostics(): Group-wise anomaly detection

Examples

library(dplyr)

walmart_sales_weekly %>%
    group_by(id) %>%
    plot_anomaly_diagnostics(Date, Weekly_Sales,
                             .message = FALSE,
                             .facet_ncol = 3,
                             .ribbon_alpha = 0.25,
                             .interactive = FALSE)

Description

An interactive and scalable function for visualizing time series seasonality. Plots are available in interactive plotly (default) and static ggplot2 format.

Usage

plot_seasonal_diagnostics(
  .data,
  .date_var,
  .value,
  .facet_vars = NULL,
  .feature_set = "auto",
  .geom = c("boxplot", "violin"),
  .geom_color = "#2c3e50",
  .geom_outlier_color = "#2c3e50",
  .title = "Seasonal Diagnostics",
  .x_lab = "",
  .y_lab = "",
  .interactive = TRUE
)

Arguments

Details

Automatic Feature Selection

Internal calculations are performed to detect a sub-range of features to include useing the following logic:

• The minimum feature is selected based on the median difference between consecutive timestamps

• The maximum feature is selected based on having 2 full periods.

Example: Hourly timestamp data that lasts more than 2 weeks will have the following features: "hour", "wday.lbl", and "week".

Scalable with Grouped Data Frames

This function respects grouped data.frame and tibbles that were made with dplyr::group_by().

For grouped data, the automatic feature selection returned is a collection of all features within the sub-groups. This means extra features are returned even though they may be meaningless for some of the groups.

Transformations

The .value parameter respects transformations (e.g. .value = log(sales)).

Value

A plotly or ggplot2 visualization

Examples

library(dplyr)

# ---- MULTIPLE FREQUENCY ----
# Taylor 30-minute dataset from forecast package
taylor_30_min

# Visualize series
taylor_30_min %>%
    plot_time_series(date, value, .interactive = FALSE)

# Visualize seasonality
taylor_30_min %>%
    plot_seasonal_diagnostics(date, value, .interactive = FALSE)

# ---- GROUPED EXAMPLES ----
# m4 hourly dataset
m4_hourly

# Visualize series
m4_hourly %>%
    group_by(id) %>%
    plot_time_series(date, value, .facet_scales = "free", .interactive = FALSE)

# Visualize seasonality
m4_hourly %>%
    group_by(id) %>%
    plot_seasonal_diagnostics(date, value, .interactive = FALSE)

Description

An interactive and scalable function for visualizing time series STL Decomposition. Plots are available in interactive plotly (default) and static ggplot2 format.

Usage

plot_stl_diagnostics(
  .data,
  .date_var,
  .value,
  .facet_vars = NULL,
  .feature_set = c("observed", "season", "trend", "remainder", "seasadj"),
  .frequency = "auto",
  .trend = "auto",
  .message = TRUE,
  .facet_scales = "free",
  .line_color = "#2c3e50",
  .line_size = 0.5,
  .line_type = 1,
  .line_alpha = 1,
  .title = "STL Diagnostics",
  .x_lab = "",
  .y_lab = "",
  .interactive = TRUE
)

Arguments

Details

The plot_stl_diagnostics() function generates a Seasonal-Trend-Loess decomposition. The function is "tidy" in the sense that it works on data frames and is designed to work with dplyr groups.

STL method:

The STL method implements time series decomposition using the underlying stats::stl(). The decomposition separates the "season" and "trend" components from the "observed" values leaving the "remainder".

Frequency & Trend Selection

The user can control two parameters: .frequency and .trend.

1. The .frequency parameter adjusts the "season" component that is removed from the "observed" values.

2. The .trend parameter adjusts the trend window (t.window parameter from stl()) that is used.

The user may supply both .frequency and .trend as time-based durations (e.g. "6 weeks") or numeric values (e.g. 180) or "auto", which automatically selects the frequency and/or trend based on the scale of the time series.

Value

A plotly or ggplot2 visualization

Examples

library(dplyr)

# ---- SINGLE TIME SERIES DECOMPOSITION ----
m4_hourly %>%
    filter(id == "H10") %>%
    plot_stl_diagnostics(
        date, value,
        # Set features to return, desired frequency and trend
        .feature_set = c("observed", "season", "trend", "remainder"),
        .frequency   = "24 hours",
        .trend       = "1 week",
        .interactive = FALSE)


# ---- GROUPS ----
m4_hourly %>%
    group_by(id) %>%
    plot_stl_diagnostics(
        date, value,
        .feature_set = c("observed", "season", "trend"),
        .interactive = FALSE)

Description

A workhorse time-series plotting function that generates interactive plotly plots, consolidates 20+ lines of ggplot2 code, and scales well to many time series.

Usage

plot_time_series(
  .data,
  .date_var,
  .value,
  .color_var = NULL,
  .facet_vars = NULL,
  .facet_ncol = 1,
  .facet_nrow = 1,
  .facet_scales = "free_y",
  .facet_dir = "h",
  .facet_collapse = FALSE,
  .facet_collapse_sep = " ",
  .facet_strip_remove = FALSE,
  .line_color = "#2c3e50",
  .line_size = 0.5,
  .line_type = 1,
  .line_alpha = 1,
  .y_intercept = NULL,
  .y_intercept_color = "#2c3e50",
  .x_intercept = NULL,
  .x_intercept_color = "#2c3e50",
  .smooth = TRUE,
  .smooth_period = "auto",
  .smooth_message = FALSE,
  .smooth_span = NULL,
  .smooth_degree = 2,
  .smooth_color = "#3366FF",
  .smooth_size = 1,
  .smooth_alpha = 1,
  .legend_show = TRUE,
  .title = "Time Series Plot",
  .x_lab = "",
  .y_lab = "",
  .color_lab = "Legend",
  .interactive = TRUE,
  .plotly_slider = FALSE,
  .trelliscope = FALSE,
  .trelliscope_params = list()
)

Arguments

Details

plot_time_series() is a scalable function that works with both ungrouped and grouped data.frame objects (and tibbles!).

Interactive by Default

plot_time_series() is built for exploration using:

• Interactive Plots: plotly (default) - Great for exploring!

• Static Plots: ggplot2 (set .interactive = FALSE) - Great for PDF Reports

By default, an interactive plotly visualization is returned.

Scalable with Facets & Dplyr Groups

plot_time_series() returns multiple time series plots using ggplot2 facets:

• group_by() - If groups are detected, multiple facets are returned

• plot_time_series(.facet_vars) - You can manually supply facets as well.

Can Transform Values just like ggplot

The .values argument accepts transformations just like ggplot2. For example, if you want to take the log of sales you can use a call like plot_time_series(date, log(sales)) and the log transformation will be applied.

Smoother Period / Span Calculation

The .smooth = TRUE option returns a smoother that is calculated based on either:

1. A .smooth_period: Number of observations

2. A .smooth_span: A percentage of observations

By default, the .smooth_period is automatically calculated using 75% of the observertions. This is the same as geom_smooth(method = "loess", span = 0.75).

A user can specify a time-based window (e.g. .smooth_period = "1 year") or a numeric value (e.g. smooth_period = 365).

Time-based windows return the median number of observations in a window using tk_get_trend().

Value

A static ggplot2 plot or an interactive plotly plot

Examples

library(dplyr)
library(lubridate)

# Works with individual time series
FANG %>%
    filter(symbol == "FB") %>%
    plot_time_series(date, adjusted, .interactive = FALSE)

# Works with groups
FANG %>%
    group_by(symbol) %>%
    plot_time_series(date, adjusted,
                     .facet_ncol  = 2,     # 2-column layout
                     .interactive = FALSE)

# Can also group inside & use .color_var
FANG %>%
    mutate(year = year(date)) %>%
    plot_time_series(date, adjusted,
                     .facet_vars     = c(symbol, year), # add groups/facets
                     .color_var      = year,            # color by year
                     .facet_ncol     = 4,
                     .facet_scales   = "free",
                     .facet_collapse = TRUE,  # combine group strip text into 1 line
                     .interactive    = FALSE)

# Can apply transformations to .value or .color_var
# - .value = log(adjusted)
# - .color_var = year(date)
FANG %>%
    plot_time_series(date, log(adjusted),
                     .color_var    = year(date),
                     .facet_vars   = contains("symbol"),
                     .facet_ncol   = 2,
                     .facet_scales = "free",
                     .y_lab        = "Log Scale",
                     .interactive  = FALSE)

Description

A boxplot function that generates interactive plotly plots for time series.

Usage

plot_time_series_boxplot(
  .data,
  .date_var,
  .value,
  .period,
  .color_var = NULL,
  .facet_vars = NULL,
  .facet_ncol = 1,
  .facet_nrow = 1,
  .facet_scales = "free_y",
  .facet_dir = "h",
  .facet_collapse = FALSE,
  .facet_collapse_sep = " ",
  .facet_strip_remove = FALSE,
  .line_color = "#2c3e50",
  .line_size = 0.5,
  .line_type = 1,
  .line_alpha = 1,
  .y_intercept = NULL,
  .y_intercept_color = "#2c3e50",
  .smooth = TRUE,
  .smooth_func = ~mean(.x, na.rm = TRUE),
  .smooth_period = "auto",
  .smooth_message = FALSE,
  .smooth_span = NULL,
  .smooth_degree = 2,
  .smooth_color = "#3366FF",
  .smooth_size = 1,
  .smooth_alpha = 1,
  .legend_show = TRUE,
  .title = "Time Series Plot",
  .x_lab = "",
  .y_lab = "",
  .color_lab = "Legend",
  .interactive = TRUE,
  .plotly_slider = FALSE,
  .trelliscope = FALSE,
  .trelliscope_params = list()
)

Arguments

Details

plot_time_series_boxplot() is a scalable function that works with both ungrouped and grouped data.frame objects (and tibbles!).

Interactive by Default

plot_time_series_boxplot() is built for exploration using:

• Interactive Plots: plotly (default) - Great for exploring!

• Static Plots: ggplot2 (set .interactive = FALSE) - Great for PDF Reports

By default, an interactive plotly visualization is returned.

Scalable with Facets & Dplyr Groups

plot_time_series_boxplot() returns multiple time series plots using ggplot2 facets:

• group_by() - If groups are detected, multiple facets are returned

• plot_time_series_boxplot(.facet_vars) - You can manually supply facets as well.

Can Transform Values just like ggplot

The .values argument accepts transformations just like ggplot2. For example, if you want to take the log of sales you can use a call like plot_time_series_boxplot(date, log(sales)) and the log transformation will be applied.

Smoother Period / Span Calculation

The .smooth = TRUE option returns a smoother that is calculated based on either:

1. A .smooth_func: The method of aggregation. Usually an aggregation like mean is used. The purrr-style function syntax can be used to apply complex functions.

2. A .smooth_period: Number of observations

3. A .smooth_span: A percentage of observations

By default, the .smooth_period is automatically calculated using 75% of the observertions. This is the same as geom_smooth(method = "loess", span = 0.75).

A user can specify a time-based window (e.g. .smooth_period = "1 year") or a numeric value (e.g. smooth_period = 365).

Time-based windows return the median number of observations in a window using tk_get_trend().

Value

A static ggplot2 plot or an interactive plotly plot

Examples

library(dplyr)
library(lubridate)

# Works with individual time series
FANG %>%
    filter(symbol == "FB") %>%
    plot_time_series_boxplot(
        date, adjusted,
        .period      = "3 month",
        .interactive = FALSE)

# Works with groups
FANG %>%
    group_by(symbol) %>%
    plot_time_series_boxplot(
        date, adjusted,
        .period      = "3 months",
        .facet_ncol  = 2,     # 2-column layout
        .interactive = FALSE)


# Can also group inside & use .color_var
FANG %>%
    mutate(year = year(date)) %>%
    plot_time_series_boxplot(
        date, adjusted,
        .period      = "3 months",
        .facet_vars   = c(symbol, year), # add groups/facets
        .color_var    = year,            # color by year
        .facet_ncol   = 4,
        .facet_scales = "free",
        .interactive  = FALSE)


# Can apply transformations to .value or .color_var
# - .value = log(adjusted)
# - .color_var = year(date)
FANG %>%
    plot_time_series_boxplot(
        date, log(adjusted),
        .period      = "3 months",
        .color_var    = year(date),
        .facet_vars   = contains("symbol"),
        .facet_ncol   = 2,
        .facet_scales = "free",
        .y_lab        = "Log Scale",
        .interactive  = FALSE)

# Can adjust the smoother
FANG %>%
    group_by(symbol) %>%
    plot_time_series_boxplot(
        date, adjusted,
        .period           = "3 months",
        .smooth           = TRUE,
        .smooth_func      = median,    # Smoother function
        .smooth_period    = "5 years", # Smoother Period
        .facet_ncol       = 2,
        .interactive      = FALSE)

Description

The plot_time_series_cv_plan() function provides a visualization for a time series resample specification (rset) of either rolling_origin or time_series_cv class.

Usage

plot_time_series_cv_plan(
  .data,
  .date_var,
  .value,
  ...,
  .smooth = FALSE,
  .title = "Time Series Cross Validation Plan"
)

Arguments

Details

Resample Set

A resample set is an output of the timetk::time_series_cv() function or the rsample::rolling_origin() function.

Value

Returns a static ggplot or interactive plotly object depending on whether or not .interactive is FALSE or TRUE, respectively.

See Also

• time_series_cv() and rsample::rolling_origin() - Functions used to create time series resample specfications.

• plot_time_series_cv_plan() - The plotting function used for visualizing the time series resample plan.

Examples

library(dplyr)
library(rsample)

FB_tbl <- FANG %>%
    filter(symbol == "FB") %>%
    select(symbol, date, adjusted)

resample_spec <- time_series_cv(
    FB_tbl,
    initial = "1 year",
    assess  = "6 weeks",
    skip    = "3 months",
    lag     = "1 month",
    cumulative  = FALSE,
    slice_limit = 6
)

resample_spec %>% tk_time_series_cv_plan()

resample_spec %>%
    tk_time_series_cv_plan() %>%
    plot_time_series_cv_plan(
        date, adjusted, # date variable and value variable
        # Additional arguments passed to plot_time_series(),
        .facet_ncol = 2,
        .line_alpha = 0.5,
        .interactive = FALSE
    )

Description

A wrapper for stats::lm() that overlays a linear regression fitted model over a time series, which can help show the effect of feature engineering

Usage

plot_time_series_regression(
  .data,
  .date_var,
  .formula,
  .show_summary = FALSE,
  ...
)

Arguments

Details

plot_time_series_regression() is a scalable function that works with both ungrouped and grouped data.frame objects (and tibbles!).

Time Series Formula

The .formula uses stats::lm() to apply a linear regression, which is used to visualize the effect of feature engineering on a time series.

• The left-hand side of the formula is used as the y-axis value.

• The right-hand side of the formula is used to develop the linear regression model.

Interactive by Default

plot_time_series_regression() is built for exploration using:

• Interactive Plots: plotly (default) - Great for exploring!

• Static Plots: ggplot2 (set .interactive = FALSE) - Great for PDF Reports

By default, an interactive plotly visualization is returned.

Scalable with Facets & Dplyr Groups

plot_time_series_regression() returns multiple time series plots using ggplot2 facets:

• group_by() - If groups are detected, multiple facets are returned

• plot_time_series_regression(.facet_vars) - You can manually supply facets as well.

Value

A static ggplot2 plot or an interactive plotly plot

Examples

library(dplyr)
library(lubridate)

# ---- SINGLE SERIES ----
m4_monthly %>%
    filter(id == "M750") %>%
    plot_time_series_regression(
        .date_var     = date,
        .formula      = log(value) ~ as.numeric(date) + month(date, label = TRUE),
        .show_summary = TRUE,
        .facet_ncol   = 2,
        .interactive  = FALSE
    )


# ---- GROUPED SERIES ----
m4_monthly %>%
    group_by(id) %>%
    plot_time_series_regression(
        .date_var    = date,
        .formula     = log(value) ~ as.numeric(date) + month(date, label = TRUE),
        .facet_ncol  = 2,
        .interactive = FALSE
    )

Description

Get and modify the Time Scale Template

Usage

set_tk_time_scale_template(.data)

get_tk_time_scale_template()

tk_time_scale_template()

Arguments

Details

Used to get and set the time scale template, which is used by tk_get_frequency() and tk_get_trend() when period = "auto".

The predefined template is stored in a function tk_time_scale_template(). This is the default used by timetk.

Changing the Default Template

• You can access the current template with get_tk_time_scale_template().

• You can modify the current template with set_tk_time_scale_template().

Value

• get_tk_time_scale_template(): Returns tibble containing the time scale template information.

• set_tk_time_scale_template(): Returns nothing.

See Also

• Automated Frequency and Trend Calculation: tk_get_frequency(), tk_get_trend()

Examples

get_tk_time_scale_template()

set_tk_time_scale_template(tk_time_scale_template())

Description

Applies a dplyr slice inside a time-based period (window).

Usage

slice_period(.data, ..., .date_var, .period = "1 day")

Arguments

Value

A tibble or data.frame

See Also

Time-Based dplyr functions:

• summarise_by_time() - Easily summarise using a date column.

• mutate_by_time() - Simplifies applying mutations by time windows.

• pad_by_time() - Insert time series rows with regularly spaced timestamps

• filter_by_time() - Quickly filter using date ranges.

• filter_period() - Apply filtering expressions inside periods (windows)

• slice_period() - Apply slice inside periods (windows)

• condense_period() - Convert to a different periodicity

• between_time() - Range detection for date or date-time sequences.

• slidify() - Turn any function into a sliding (rolling) function

Examples

# Libraries
library(dplyr)

# First 5 observations in each month
m4_daily %>%
    group_by(id) %>%
    slice_period(1:5, .period = "1 month")

# Last observation in each month
m4_daily %>%
    group_by(id) %>%
    slice_period(n(), .period = "1 month")

Description

slidify returns a rolling (sliding) version of the input function, with a rolling (sliding) .period specified by the user.

Usage

slidify(
  .f,
  .period = 1,
  .align = c("center", "left", "right"),
  .partial = FALSE,
  .unlist = TRUE
)

Arguments

Details

The slidify() function is almost identical to tibbletime::rollify() with 3 improvements:

1. Alignment ("center", "left", "right")

2. Partial windows are allowed

3. Uses slider under the hood, which improves speed and reliability by implementing code at C++ level

Make any function a Sliding (Rolling) Function

slidify() turns a function into a sliding version of itself for use inside of a call to dplyr::mutate(), however it works equally as well when called from purrr::map().

Because of it's intended use with dplyr::mutate(), slidify creates a function that always returns output with the same length of the input

Alignment

Rolling / Sliding functions generate .period - 1 fewer values than the incoming vector. Thus, the vector needs to be aligned. Alignment of the vector follows 3 types:

• center (default): NA or .partial values are divided and added to the beginning and end of the series to "Center" the moving average. This is common in Time Series applications (e.g. denoising).

• left: NA or .partial values are added to the end to shift the series to the Left.

• right: NA or .partial values are added to the beginning to shift the series to the Right. This is common in Financial Applications (e.g moving average cross-overs).

Allowing Partial Windows

A key improvement over tibbletime::slidify() is that timetk::slidify() implements .partial rolling windows. Just set .partial = TRUE.

Value

A function with the rolling/sliding conversion applied.

References

• The Tibbletime R Package by Davis Vaughan, which includes the original rollify() Function

See Also

Transformation Functions:

• slidify_vec() - A simple vectorized function for applying summary functions to rolling windows.

Augmentation Functions (Add Rolling Multiple Columns):

• tk_augment_slidify() - For easily adding multiple rolling windows to you data

Slider R Package:

• slider::pslide() - The workhorse function that powers timetk::slidify()

Examples

library(dplyr)

FB <- FANG %>% dplyr::filter(symbol == "FB")


# --- ROLLING MEAN (SINGLE ARG EXAMPLE) ---

# Turn the normal mean function into a rolling mean with a 5 row .period
mean_roll_5 <- slidify(mean, .period = 5, .align = "right")

FB %>%
    mutate(rolling_mean_5 = mean_roll_5(adjusted))

# Use `partial = TRUE` to allow partial windows (those with less than the full .period)
mean_roll_5_partial <- slidify(mean, .period = 5, .align = "right", .partial = TRUE)

FB %>%
    mutate(rolling_mean_5 = mean_roll_5_partial(adjusted))

# There's nothing stopping you from combining multiple rolling functions with
# different .period sizes in the same mutate call

mean_roll_10 <- slidify(mean, .period = 10, .align = "right")

FB %>%
    select(symbol, date, adjusted) %>%
    mutate(
        rolling_mean_5  = mean_roll_5(adjusted),
        rolling_mean_10 = mean_roll_10(adjusted)
    )

# For summary operations like rolling means, we can accomplish large-scale
# multi-rolls with tk_augment_slidify()

FB %>%
    select(symbol, date, adjusted) %>%
    tk_augment_slidify(
        adjusted, .period = 5:10, .f = mean, .align = "right",
        .names = stringr::str_c("MA_", 5:10)
    )

# --- GROUPS AND ROLLING ----

# One of the most powerful things about this is that it works with
# groups since `mutate` is being used

mean_roll_3 <- slidify(mean, .period = 3, .align = "right")

FANG %>%
    group_by(symbol) %>%
    mutate(mean_roll = mean_roll_3(adjusted)) %>%
    slice(1:5)


# --- ROLLING CORRELATION (MULTIPLE ARG EXAMPLE) ---

# With 2 args, use the purrr syntax of ~ and .x, .y
# Rolling correlation example
cor_roll <- slidify(~cor(.x, .y), .period = 5, .align = "right")

FB %>%
    mutate(running_cor = cor_roll(adjusted, open))

# With >2 args, create an anonymous function with >2 args or use
# the purrr convention of ..1, ..2, ..3 to refer to the arguments
avg_of_avgs <- slidify(
    function(x, y, z) (mean(x) + mean(y) + mean(z)) / 3,
    .period = 10,
    .align = "right"
)

# Or
avg_of_avgs <- slidify(
    ~(mean(..1) + mean(..2) + mean(..3)) / 3,
    .period = 10,
    .align  = "right"
)

FB %>%
    mutate(avg_of_avgs = avg_of_avgs(open, high, low))

# Optional arguments MUST be passed at the creation of the rolling function
# Only data arguments that are "rolled over" are allowed when calling the
# rolling version of the function
FB$adjusted[1] <- NA

roll_mean_na_rm <- slidify(~mean(.x, na.rm = TRUE), .period = 5, .align = "right")

FB %>%
    mutate(roll_mean = roll_mean_na_rm(adjusted))


# --- ROLLING REGRESSIONS ----

# Rolling regressions are easy to implement using `.unlist = FALSE`
lm_roll <- slidify(~lm(.x ~ .y), .period = 90, .unlist = FALSE, .align = "right")

FB %>%
    tidyr::drop_na() %>%
    mutate(numeric_date = as.numeric(date)) %>%
    mutate(rolling_lm = lm_roll(adjusted, numeric_date)) %>%
    filter(!is.na(rolling_lm))

Description

slidify_vec() applies a summary function to a rolling sequence of windows.

Usage

slidify_vec(
  .x,
  .f,
  ...,
  .period = 1,
  .align = c("center", "left", "right"),
  .partial = FALSE
)

Arguments

Details

The slidify_vec() function is a wrapper for slider::slide_vec() with parameters simplified "center", "left", "right" alignment.

Vector Length In == Vector Length Out

NA values or .partial values are always returned to ensure the length of the return vector is the same length of the incoming vector. This ensures easier use with dplyr::mutate().

Alignment

Rolling functions generate .period - 1 fewer values than the incoming vector. Thus, the vector needs to be aligned. Alignment of the vector follows 3 types:

• Center: NA or .partial values are divided and added to the beginning and end of the series to "Center" the moving average. This is common for de-noising operations. See also ⁠[smooth_vec()]⁠ for LOESS without NA values.

• Left: NA or .partial values are added to the end to shift the series to the Left.

• Right: NA or .partial values are added to the beginning to shif the series to the Right. This is common in Financial Applications such as moving average cross-overs.

Partial Values

• The advantage to using .partial values vs NA padding is that the series can be filled (good for time-series de-noising operations).

• The downside to partial values is that the partials can become less stable at the regions where incomplete windows are used.

If instability is not desirable for de-noising operations, a suitable alternative is smooth_vec(), which implements local polynomial regression.

Value

A numeric vector

References

• Slider R Package by Davis Vaughan

See Also

Modeling and More Complex Rolling Operations:

• step_slidify() - Roll apply for tidymodels modeling

• tk_augment_slidify() - Add many rolling columns group-wise

• slidify() - Turn any function into a rolling function. Great for rolling cor, rolling regression, etc.

• For more complex rolling operations, check out the slider R package.

Vectorized Transformation Functions:

• Box Cox Transformation: box_cox_vec()

• Lag Transformation: lag_vec()

• Differencing Transformation: diff_vec()

• Rolling Window Transformation: slidify_vec()

• Loess Smoothing Transformation: smooth_vec()

• Fourier Series: fourier_vec()

• Missing Value Imputation for Time Series: ts_impute_vec()

Examples

library(dplyr)
library(ggplot2)

# Training Data
FB_tbl <- FANG %>%
    filter(symbol == "FB") %>%
    select(symbol, date, adjusted)

# ---- FUNCTION FORMAT ----
# - The `.f = mean` function is used. Argument `na.rm = TRUE` is passed as ...
FB_tbl %>%
    mutate(adjusted_30_ma = slidify_vec(
        .x      = adjusted,
        .period = 30,
        .f      = mean,
        na.rm   = TRUE,
        .align  = "center")) %>%
        ggplot(aes(date, adjusted)) +
        geom_line() +
        geom_line(aes(y = adjusted_30_ma), color = "blue", na.rm = TRUE)

# ---- FORMULA FORMAT ----
# - Anonymous function `.f = ~ mean(., na.rm = TRUE)` is used
FB_tbl %>%
    mutate(adjusted_30_ma = slidify_vec(
        .x      = adjusted,
        .period = 30,
        .f      = ~ mean(., na.rm = TRUE),
        .align  = "center")) %>%
        ggplot(aes(date, adjusted)) +
        geom_line() +
        geom_line(aes(y = adjusted_30_ma), color = "blue", na.rm = TRUE)

# ---- PARTIAL VALUES ----
# - set `.partial = TRUE`
FB_tbl %>%
    mutate(adjusted_30_ma = slidify_vec(
        .x       = adjusted,
        .f       = ~ mean(., na.rm = TRUE),
        .period  = 30,
        .align   = "center",
        .partial = TRUE)) %>%
        ggplot(aes(date, adjusted)) +
        geom_line() +
        geom_line(aes(y = adjusted_30_ma), color = "blue")

# ---- Loess vs Moving Average ----
# - Loess: Using `.degree = 0` to make less flexible. Comparable to a moving average.

FB_tbl %>%
    mutate(
        adjusted_loess_30 = smooth_vec(adjusted, period = 30, degree = 0),
        adjusted_ma_30    = slidify_vec(adjusted, .f = mean,
                                           .period = 30, .partial = TRUE)
    ) %>%
    ggplot(aes(date, adjusted)) +
    geom_line() +
    geom_line(aes(y = adjusted_loess_30), color = "red") +
    geom_line(aes(y = adjusted_ma_30), color = "blue") +
    labs(title = "Loess vs Moving Average")

Description

smooth_vec() applies a LOESS transformation to a numeric vector.

Usage

smooth_vec(x, period = 30, span = NULL, degree = 2)

Arguments

Details

Benefits:

• When using period, the effect is similar to a moving average without creating missing values.

• When using span, the effect is to detect the trend in a series using a percentage of the total number of observations.

Loess Smoother Algorithm This function is a simplified wrapper for the stats::loess() with a modification to set a fixed period rather than a percentage of data points via a span.

Why Period vs Span? The period is fixed whereas the span changes as the number of observations change.

When to use Period? The effect of using a period is similar to a Moving Average where the Window Size is the Fixed Period. This helps when you are trying to smooth local trends. If you want a 30-day moving average, specify period = 30.

When to use Span? Span is easier to specify when you want a Long-Term Trendline where the window size is unknown. You can specify span = 0.75 to locally regress using a window of 75% of the data.

Value

A numeric vector

See Also

Loess Modeling Functions:

• step_smooth() - Recipe for tidymodels workflow

Additional Vector Functions:

• Box Cox Transformation: box_cox_vec()

• Lag Transformation: lag_vec()

• Differencing Transformation: diff_vec()

• Rolling Window Transformation: slidify_vec()

• Loess Smoothing Transformation: smooth_vec()

• Fourier Series: fourier_vec()

• Missing Value Imputation for Time Series: ts_impute_vec()

Examples

library(dplyr)
library(ggplot2)

# Training Data
FB_tbl <- FANG %>%
    filter(symbol == "FB") %>%
    select(symbol, date, adjusted)

# ---- PERIOD ----

FB_tbl %>%
    mutate(adjusted_30 = smooth_vec(adjusted, period = 30, degree = 2)) %>%
    ggplot(aes(date, adjusted)) +
    geom_line() +
    geom_line(aes(y = adjusted_30), color = "red")

# ---- SPAN ----

FB_tbl %>%
    mutate(adjusted_30 = smooth_vec(adjusted, span = 0.75, degree = 2)) %>%
    ggplot(aes(date, adjusted)) +
    geom_line() +
    geom_line(aes(y = adjusted_30), color = "red")

# ---- Loess vs Moving Average ----
# - Loess: Using `degree = 0` to make less flexible. Comperable to a moving average.

FB_tbl %>%
    mutate(
        adjusted_loess_30 = smooth_vec(adjusted, period = 30, degree = 0),
        adjusted_ma_30    = slidify_vec(adjusted, .period = 30,
                                        .f = mean, .partial = TRUE)
    ) %>%
    ggplot(aes(date, adjusted)) +
    geom_line() +
    geom_line(aes(y = adjusted_loess_30), color = "red") +
    geom_line(aes(y = adjusted_ma_30), color = "blue") +
    labs(title = "Loess vs Moving Average")