Measurement, Meaning and Purpose: Exploring the M-Lab NDT Dataset

44 Pages Posted: 3 Aug 2021 Last revised: 11 Aug 2021

See all articles by David D. Clark

David D. Clark

MIT CSAIL

Sara Wedeman

Massachusetts Institute of Technology - CSAIL

Date Written: August 2, 2021

Abstract

The speed of a data transfer over the Internet connection is a measure of great interest. It can directly influence quality of experience, it can serve as a measure of equitable access, and it can reveal whether or not the speed matches that which was advertised. One of earliest and longest-standing tools to contain a speed test is the Network Diagnostic Tool (NDT), currently supported by Measurement Lab (M-Lab). NDT, in its various forms, has been used across the globe since 2003. There are billions of measurements archived by M-Lab, with a rich collection of metadata for each measurement. This data allows an in-depth analysis of each measurement, and potentially supports analysis across aggregates of measurements.

The original purpose of NDT was diagnostic: why is my connection operating as it is? However, the archive of this data invites its use in aggregate form to draw conclusions about the overall behavior of the Internet. Such use, however, is confounded by the fact that the individual measurements are triggered by users for a range of reasons: simple curiosity, debugging, anger, bragging rights, or automated checking of operational status (in some cases as often as once a minute). NDT measures network speed, but it equally - if indirectly - measures human behavior.

The goals of this paper are to explore the archived NDT data in order to provide insights about how it can be interpreted; and to distinguish between appropriate uses of the data from uses that may lead to unwarranted conclusions.

Keywords: Internet measurement, Internet speed test, network diagnostic tool, measurement lab

Suggested Citation

Clark, David D. and Wedeman, Sara, Measurement, Meaning and Purpose: Exploring the M-Lab NDT Dataset (August 2, 2021). TPRC49: The 49th Research Conference on Communication, Information and Internet Policy, Available at SSRN: https://ssrn.com/abstract=3898339 or http://dx.doi.org/10.2139/ssrn.3898339

David D. Clark (Contact Author)

MIT CSAIL ( email )

Stata Center
Cambridge, MA 02142
United States
617-253-6003 (Phone)

Sara Wedeman

Massachusetts Institute of Technology - CSAIL ( email )

32 Vassar Street
Cambridge, MA 02139
United States

Do you have negative results from your research you’d like to share?

Paper statistics

Downloads
394
Abstract Views
2,134
Rank
137,540
PlumX Metrics