OpenPack Dataset

What is OpenPack dataset?

OpenPacking Dataset is a new large-scale multi-modal dataset of packing processes. OpenPack is an open access logistics-dataset for human activity recognition, which contains 53 hours of human movement data with 20,129 instances of work operation annotation. In addition, OpenPack dataset contains data from IoT-enabled devices and rich metadata such as error annotation and order information.

Characterisitcs

53H+

Large Scale activity dataset in Industrial Domain

OpenPack is the first large-scale dataset for packaging work recognition. 16 distinct subjects packed 3956 items in 2048 shipping boxes in total and the total recording length is more than 53 hours. The work operations and actions were annotated by the professional annotators. The total number of annotated work operations and actions are 20,129 and 52,529 respectively.

9
+ IoT

Rich Modalities - Vision + Wearable + IoT Device

OpenPack provides 9 data modalities including acceleration, gyroscope, quaternion, blood volume pulse (BVP), and electrodermal activity (EDA) data, as well as keypoints, a LiDAR point cloud, and depth images. As for the wearable sensor modality, we collected data with 4 IMUs and 2 E4 sensors. For the vision modality, 2 depth cameras (front view and top view) and 1 LiDAR sensor were installed to our environment. In addition, we collected data from IoT devices which are sparse but have strong relation with specific activity classes.

META

Rich Meta Data

OpenPack also provides a rich set of metadata such as subject’s experience in packaging work and physical traits of subjects, enabling designs of various research tasks such as assessment of worker’s performance in addition to basic work activity recognition. In addition, we made a list of irregular activities which can be used for developing normal detection technologies.

Sample Video

Acitivty Classes

Work Operations (10 classes)

ImageIDOperation
100Picking
200Relocate Item Label
300Assemble Box
400Insert Items
500Close Box
600Attach Box Label
700Scan Label
800Attach Shipping Label
900Put on Back Table
1000Fill out Order
8100Null

Download

OpenPack dataset is available in 3 repositories. Different repositories provide different sets of data with different licenses. Please go to openpack-dataset @GitHub and follow the instructions.

Sample Data

You can see samples of CSVs and images without downloading them. Sample data (U0209 - Session 5) is available on GitHub (openpack-dataset/data/openpack).

Preprocessed Dataset

Sensors used in the OpenPack dataset have different sampling rates. Therefore, each sensor is stored in a separate file. When you use them, you have to combine them using timestamps assosiated with each record. But we understand that it's not easy for the new users.

Therefore, we prepared pre-processed dataset for the quick trial. IMU data from 4 sensors and work operation labels are combined into one CSV file. This preprocessd dataset is available on zenodo (preprocessed-IMU-with-operation-labels.zip) (Click to Download [515MB]).

For more details, see data-stream/preprocessed.

Full Dataset

See DOWNLOAD.md (@GitHub:openpack-dataset).

Download Commands

Below is a command to download data from zenodo.

# Get a download script.
$ curl -o download_from_zenodo.sh \
  https://raw.githubusercontent.com/open-pack/openpack-dataset/main/release/v1.0.0/download_from_zenodo.sh 
# Create dataset root directory.
$ export DATASET_ROOTDIR_PATH=./data/datasets
$ mkdir -p $DATASET_ROOTDIR_PATH
# Download the dataset.
$ bash download_from_zenodo.sh $DATASET_ROOTDIR_PATH

Terms of Use (Licence)

The OpenPack dataset consists of RGB data (OpenPack Dataset (+RGB)) and other data (OpenPack Dataset). The dataset without RGB data will continue to be provided under "CC BY-NC-SA 4.0". On the other hand, the dataset including RGB data will be provided under a different license for privacy and ethical considerations, as described below. Please check the license of the dataset you use before using it.

OpenPack Dataset

OpenPack Dataset by OpenPack Team is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License. This means that you must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use. You may not use the material for commercial purposes.
Creative Commons License

OpenPack Dataset (+RGB)

“OpenPack Dataset (+RGB)” is one of the version that includes RGB data. OpenPack Dataset (+RGB) is licenced under “OpenPack Dataset (+RGB) License” . This means that you may not redistribute this data for any porpose. Also, the usage of this dataset is limited to the academic research only. Use of commercial purpose is prohibited. See the OPENPACK_DATASET_RGB_LICENSE.md (@GitHub:openpack-dataset) for the full license conditions.

Citation

When you use the OpenPack dataset or code please cite the following paper. (Updated on Jan 28 2024):

Naoya Yoshimura, Jaime Morales, Takuya Maekawa, Takahiro Hara, “OpenPack: A Large-scale Dataset for Recognizing Packaging Works in IoT-enabled Logistic Environments”. Proceedings of IEEE International Conference on Pervasive Computing and Communications (2024).

Tutorials

Notebooks (Colab / Jupyter Notebook)

When you click on the link, a notebook will open in Google Colaboratory. A warning will appear when you run the first cell, but press "continue" to proceed. It is recommended to download the dataset to Google Drive when you run the tutorials on Google Colab. Before starting tutorials, please follow the instructions in the first tutorial (Tutorial - Download OpenPack Dataset to Google Drive) and download the OpenPack dataset from zenodo to your Google Drive.

GitHub

OpenPack Dataset

Dataset Documentations

GitHub

OpenPack Toolkit

Dataset Utilities

GitHub

OpenPack Torch

Sample Code (PyTorch)

YouTube

OpenPack Dataset Channel

Sample Videos and a Tutorial Recording (Japanese)

Paper

Preprint

OpenPack: A Large-scale Dataset for Recognizing Packaging Works in IoT-enabled Logistic Environments

N. Yoshimura, J. Morales, T. Maekawa, T. Hara

CoRRabs/2212.11152 (2022)

In this study, we introduce a new large-scale dataset for packaging work recognition called OpenPack. OpenPack contains 53.8 hours of multimodal sensor data, including keypoints, depth images, acceleration data, and readings from IoT-enabled devices (e.g., handheld barcode scanners used in work procedures), collected from 16 distinct subjects with different levels of packaging work experience.

Conference

Recent Trends in Sensor-based Activity Recognition

T. Maekawa, Q. Xia, R. Otsuka, N. Yoshimura, K. Tanigaki

Proceedings of the IEEE International Conference on Mobile Data Management (MDM)

This seminar introduces recent trends in sensor-based activity recognition technology. Technology to recognize human activities using sensors has been a hot topic in the field of mobile and ubiquitous computing for many years. Recent developments in deep learning and sensor technology have expanded the application of activity recognition to various domains such as industrial and natural science fields. However, because activity recognition in the new domains suffers from various real problems such as the lack of sufficient training data and complexity of target activities, new solutions have been proposed for the practical problems in applying activity recognition to real-world applications in the new domains. In this seminar, we introduce recent topics in activity recognition from the viewpoints of (1) recent trends in state-of-the-art machine learning methods for practical activity recognition, (2) recently focused domains for human activity recognition such as industrial and medical domains and their public datasets, and (3) applications of activity recognition to the natural science field, especially in animal behavior understanding.

Members

Takuya Maekawa
Osaka Univ.

Naoya Yoshimura
Osaka Univ.

Jaime Morales
Osaka Univ.

Special Thanks

K

Kana Yasuda
Annotator

C

Chikako Kawabe
Annotator

M

Makiko Otsuka
Annotator

Y

Yashodmi Kaluarachchi
Annotator

K

Keisuke Tsukamoto
Web Developper

P

Pan Guangyang
Web Developper

K

Kohei Hirata
Web Developper

Y

Yagi
Web Developper

Get Updates from OpenPack Team!

Get the latest information about the OpenPack Challenge or OpenPack Dataset on our mailing list (Google Group), Twitter and GitHub. Please subscribe and follow now!