Overview
Parenting and parent-child interactions often take place inside a black box, frequently guessed at but typically invisible to outside observers. But what goes on inside that black box is key to helping children get off to a strong, healthy start in life. The New Jersey Families Study (NJFS) unlocks that black box for the first time. It is unprecedented in its observational scale and in the granular richness of its data. No one has examined home life in the United States with this breadth, intensity, or duration.
The innovative component of the study consists of approximately 11,500 hours of in-home video with audio recordings, obtained from 21 families with young children in Mercer County, NJ that consented to having two weeks of their daily lives video recorded. The raw data are contained in roughly 504,000 discrete video clips. As attractive as this vast quantity may be, potential users are unlikely to invest the time searching through all these data to find the clips of greatest relevance. Our solution for creating a user-friendly dataset is to house the video clips in a searchable database, attach a variety of metadata to them, and then query the database using a manageable set of filters. The proposed set of filters includes household demographic and socioeconomic characteristics, room views (living room, kitchen, etc.), day and time periods, who the participants are in the clips, and activities and behaviors captured in the videos. Information is in hand for the first three of these. Automating the coding of clips for who the participants are in the videos and for behaviors and activities will involve close collaboration with our colleagues in Princeton’s computer science department around artificial intelligence, machine learning, and computer vision.
These video data are supplemented with a unique combination of survey, interview, and other video data collected during six additional points of contact with families. It was not unusual for the research team to spend three or four months with family members from the time data collection began until it ended.
Our goal is to create a user-friendly, early childhood database for a worldwide research community, while honoring promises of confidentiality made to participating NJFS families. We want the NJFS to become a community resource that is made available as soon as possible.
Access to the Data
Use of the NJFS in-home video recordings and other project data will be restricted-access, not public-use. Interested users will have to apply for permission to access the data and will need to agree to protect the data in the same way that project staff are doing. Exact rules of engagement are still being developed. The current plan is to make the data available once a searchable database has been designed, built, tested, and operationalized using filters for room views, temporal markers, and household demographic and socioeconomic information. It is anticipated that usability features of the NJFS early childhood database will continue to evolve and that improvements will continue to be made—partly in response to feedback from the user community. Information about when and how NJFS data can be accessed will be posted on the NJFS website.
Sample Video Clips
The three sample clips shown here give potential users a sense of the clarity of NJFS video images at three different resolutions (480p, 720p, and 1080p). NJFS video clips were recorded in HD using Mosquito Pro 4.0 cameras from Security Camera Warehouse (SCW). That model has been discontinued. The images here were recorded using a Deputy 4.0 v3 IP camera from SCW and are similar to the Mosquito in terms of video and audio quality.
Video 1 – 480p
Video 2 – 720p
Video 3 – 1080p
Storing the Video Clips
NJFS video data are securely stored on Citadel (citadel.princeton.edu), Princeton's new secure and compliant research data infrastructure environment introduced in 2021 by Princeton Research Computing, a consortium spearheaded by the Princeton Institute for Computational Science and Engineering and OIT Research Computing. Citadel enables researchers from anywhere in the world to handle sensitive data while taking strict measures to prevent unauthorized access. Currently, six projects at Princeton are using the Citadel environment, including the NJFS.
In the past, researchers tasked with hosting restricted data would often implement a one-off solution. For example, members of the NJFS team wishing to use the footage would have to physically visit the Data Manager's office to view it on a standalone computer.
Migrating to Citadel solved that problem by allowing secure remote access. Connections to Citadel, whose servers are locked in a cage at Princeton's High-Performance Computing Research Center, are tightly controlled. Authorized researchers can access their datasets via their desktop or laptop computers, but the files are not downloaded to these devices. Instead, they are accessed and manipulated inside virtual machines that are kept isolated.
Videos on Citadel are stored at 480p to save storage space. Customized solutions at higher resolution might be possible.
Servers locked in a cage as part of the Secure Research Infrastructure at Princeton University.
Photo: Michael Monaghan, Office of Information Technology, Princeton University
Source
Building a Searchable Database
Up to this point, three kinds of tags or metadata have been developed to associate with each video clip: room views, temporal markers, and household demographic and socioeconomic characteristics.
-
Room Views
Table X. NJFS Camera Views (with OIT Room Labels) shows the room in the family home or the camera view associated with each camera number.
The List of Room View Filters (Final) is an initial cut at a list of room view filters that would permit users to call up videos associated with a particular room in the house.
-
Temporal Markers
The filename for each video clip contains the date and time when the video recording began (Table V. Screenshot of a Sample of Video Clip Filenames in Citadel).
These data and time stamps permit a variety of filters for days and times, a few of which are shown in the associated table Temporal candidates_20211115.
-
Household Characteristics
Data on family demographic and socioeconomic characteristics come from a 5-10 minute Interest Survey that each family was asked to complete when they showed potential interest in participating in the New Jersey Families Study. The Interest Survey also contained the respondent’s name, address, and contact information. There are two versions of the Interest Survey. The original version was finalized in December 2015 and completed by families #1007 through #1042. The original version was slightly expanded with questions about spouse’s education and occupation in January 2018 as attention shifted to enrolling more families from moderate and low-income backgrounds. The revised version was administered to families #1044 through #1061. With one exception, families completed the Interest Survey online.
A respondent’s Socioeconomic Index is one measure of a family’s socioeconomic status, based on education and occupation. Socioeconomic Index scores for most of the 21 NJFS families are derived from occupation-based socioeconomic indices calculated by Song and Xie (2020) for the 1980 birth cohort using 535 occupations in the 2010 U.S. Census. The resulting index is the cohort-specific average percentile educational attainment for persons in a given occupation. Indices range between 0 and 100. Most of the respondents to the NJFS Interest Survey reported an occupation. This occupation was then matched as closely as possible to those in the 2010 Census, taking as the socioeconomic index for that occupation the index reported in Song and Xie (2020). In most cases there was a one-to-one match. Occasionally, a respondent’s reported occupation matched to several 2010 occupational codes. In an extreme case, the occupation “manager” matched to 19 occupations in the 2010 census. When there was more than one match, we averaged the Song and Xie (2020) socioeconomic indices.
Four respondents to the Interest Survey had no current occupation. Three of these four listed “stay-at-home parent” as their occupation. Each of these individuals, however, reported educational attainment. For them, we based their socioeconomic index on their percentile educational attainment rank, derived from a weighted educational attainment distribution for individuals 25-34 in the U.S. labor force for individual years 2015-2017 in the American Community Survey. We used the midpoint of the educational attainment range. For example, 6.16 percent of persons 25-34 in the labor force in the combined 2015-2017 American Community Surveys reported less than a high school degree. No respondent to the Interest Survey was in that category. But if they had been, their socioeconomic index score would have been 3.1.
Source: Song, Xi, and Yu Xie. 2020. "Occupation-Based Socioeconomic Index with Percentile Ranks." University of Pennsylvania Population Center Working Paper (PSC/PARC), 2020-59. https://repository.upenn.edu/psc_publications/59.
Numbers in parentheses indicate the number of households in each category.
Date Interest Survey completed
1 = 2016: first half (10)
2 = 2016: second half (1)
3 = 2018: first half (7)
4 = 2018: second half (3)
Date in-home recording began
1 = 2017: first half (7)
2 = 2017: second half (4)
3 = 2018: first half (2)
4 = 2018: second half (6)
5 = 2019: first half (2)
Respondent’s sex
1 = Female (19)
2 = Male (2)
Respondent’s educational attainment
1 = Less than high school (0)
2 = High school graduate (3)
3 = Some college (6)
4 = Two-year college degree (1)
5 = Four-year college degree (7)
6 = Graduate degree (4)
Respondent’s socioeconomic index
1 = 20-30 (3)
2 = 40-55 (4)
3 = 67-83 (10)
4 = 88-100 (4)
Racial background of adults
1 = Asian or Pacific Islander (1)
2 = Black or African American (6)
3 = Hispanic (1)
4 = White (6)
5 = Interracial couple (if the two adults have different racial compositions) (6)
6 = Multiracial individual or couple (if one or both adults have two or more races/ethnicities) (2)
Note: One family fell into categories 5 and 6
Two-parent household
1 = Yes (17)
2 = No (4)
Age of target child when recording began
1 = 2.0-2.5 years (2)
2 = 2.5-3.0 years (2)
3 = 3.0-3.5 years (6)
4 = 3.5-4.0 years (6)
5 = 4.0-4.5 years (3)
6 = 4.5-5.0 years (2)
Target child’s sex
1 = Female (13)
2 = Male (8)
Number of children <18 living in the home
1 = One (4)
2 = Two (8)
3 = Three (8)
4 = Four (1)
Here are the same data arrayed differently for readers who prefer a more visual representation.
Shown here are the project's two Interest Surveys: NJFS Interest Survey_20151221 and NJFS Interest Survey_20180120
The following Table Z. Items Included on NJFS Interest Surveys shows household demographic and socioeconomic information contained on the two Interest Surveys. As noted, the Interest Surveys also include personal identifiers—respondent's name, phone numbers, email address, and place of residence. This information is not shared.
Data Cleaning
Part of curating a dataset involves several forms of data cleaning. Video clips that showed OIT personnel setting up or tearing down the video recording equipment usually contained no information about family dynamics or parent-child interactions. These clips have been removed from the main video folder and placed in a separate folder. Table XI. Number of Video Clips Removed for OIT Set Up and Tear Down shows the number of OIT clips removed by household ID, camera number, and whether the removal contained data recorded during set up (SU) or tear down (TD).
Interview Data
In addition to information gathered from the in-home video recording, the project collected roughly six hours of interview data from five additional points of contact with participating NJFS families: a telephone interview, a home visit, a pre-observation interview, a debriefing interview, and a post-observation interview. Interview data provide helpful context for what is seen in the videos. But they are also enormously rich in their own right, shedding light on family values and behaviors, life goals and satisfactions, and neighborhood context. Here we summarize the kinds of questions asked during these interviews. Responses will be part of Data Release 1.0.
What comes next omits the Interest Survey and focuses on interview protocols. The Interest Survey is a short, self-administered survey that was distributed in paper during our information sessions and online via our website during our recruitment phase. It constituted an initial screening device and represents a prospective participant’s first expression of interest. It contained questions that corresponded to our criteria for inclusion. Information from the Interest Survey is not being made public, except to use some of the non-identifying responses as metadata.
Telephone Interview. Respondents who were identified as potential video ethnography participants based on the pre-screening survey were interviewed by phone. This survey was used as a more extensive and finely-tuned screening mechanism. It significantly narrowed the field of potential participants. It also provided an opportunity to inform the participants about the study details and answer any of their questions.
Home Visit. Two members of the research team came equipped with a hand-held video camera, home-made banana bread for the families, and a book for the focal child (the child got to choose among three books, “Penguin on Vacation," "Stormy Night," or “Penguin's Big Adventure,” by Salina Yoon). The purpose of the site visit was twofold: (1) to have a visual record of resources in the home that were potentially relevant to child development (for instance, electronic devices, books, toys and books for the child, and so on) and (2) to gauge how many rooms were in the home, where they were located, which rooms might be best suited for video observation, and where the best places might be to locate the recording equipment (to be as effective, unobtrusive, and out of the way as possible).
Pre-Observation Interview (with 21 Finalist Families). This semi-structured interview was conducted with the target or focal child’s primary caregiver at their place of residence. It consisted primarily of questions aimed at giving context on the parent’s own upbringing/family background, how the parent chose their particular neighborhood and home, and the current childcare routine. Importantly, the questions did not cover topics that we believed would induce desirability biases during the two-week video observation period.
In addition, during this pre-observation interview, we left an information packet containing the following documents with the family:
1. Activity diary
2. Instructions on how to use the audio recorder we left with the families to voluntarily provide context for behaviors depicted on the video recording
3. Information sign for unexpected visitors
4. New Jersey Families Study Information sheet
5. Four red placards with “stop” on them (one for each video room in the household) that families could hold up if they did not like us to view any upcoming segment. The back of these cards contained an address label that listed the number families could call if they wished to discontinue participation.
Debriefing Interview. This was a semi-structured interview conducted with all household members immediately after the end of the two-week observation period during the takedown of video cameras. It served the purpose of encouraging participants to reflect on their experiences of participating in the study while those experiences were fresh in the participants’ memory. As part of the debriefing interview, we reviewed an Activity Diary that was contained in the information packet described above.
Post-Observation Interview. Like the pre-observation interview, this was conducted with the primary caregiver in each study family at their place of residence, approximately two weeks after the end of the video observation period. Topics included parenting values, attitudes, and behaviors; parent stress and wellbeing; child social and behavioral skills; and out-of-home activities.
Future Plans and Data Releases
Data curation is underway with the goal of building a user-friendly, path-breaking, next-generation early childhood database and offering it as a community resource to the worldwide research community. Five future data releases are currently anticipated.
Release 1.0—a user-friendly searchable database for 504,000 individual video clips.
Release 2.0—metadata tags for who the participants are in the video clips and their activities and behaviors.
Release 3.0—an automated digital transcript of the audio on all 688,300 minutes of NJFS video/audio data.
Release 4.0—will use structural topic models to uncover the topics being discussed.
Release 5.0—expands the utility of the early childhood database by engaging linguists and other scholars interested in the finer points of speech analysis and synthesis.
More details can be found in Future Data Releases_20220913.