Skip to content

Development tips: Reports

Jump to bottom
Ben Silverman edited this page Jul 1, 2021 · 7 revisions

Here are some tips for creating a new report type, based on the architecture used for the Enrollment Bar Chart.

Architecture

New report form

Data flow for creating a report:

  1. User submits a form, filling out the fields from ChartFormFields.jsx (via EditReport.react.js).
  2. The form fields are sent with a POST request to the /api/v1/reports API route.
  3. The form fields are inserted into the MongoDB database dpdmongo in the collection reports.

If a new chart type is added, it should be added to this list, or if it is already in the list, the disabled prop should be removed from it.

Additionally, we have provided a form field that is currently unused, called variableMulti. It was intended to be used for the category-line chart type. However, as written, it will only work if there is only one assessment and all variables are in that assessment. If it needs to pull variables from more than one assessment, it will need to be rewritten, perhaps more like the valueLabels fields.

MongoDB structure

Reports are stored in MongoDB according to the following structure:

{
  _id: String,               // MongoDB ObjectId for the report
  reportName: String,        // Report title
  user: String,              // User ID of the creator
  readers: Array<String>,    // Array of user IDs for sharing
  created: Date,             // Date the report was created
  charts: [                  // An array of chart objects, which may be empty or contain empty fields
    {
      title: String,                   // The title of the chart
      chartType: String,               // The type of the chart (e.g. "bar")
      variableSingle: String,          // A single variable name (for multi-site charts)
      variableMulti: Array<String>,    // An array of variable names (for single-site charts)
      assessmentSingle: String,        // The name of the assessment containing the above variable(s)
      valueLabels: [                   // An array of value/label mappings
        {
          value: String,               // A single value/range, or a list of comma-separated values/ranges
          label: String,               // The label for the value(s)
	},
        ...
      ]
    },
    ...
  ]
}

View report page

Data flow for viewing a report:

  1. User selects a study or studies on Report.react.js and clicks "Generate report".
  2. A GET request is sent to /api/v1/reports/:id for each chart via the function fetchDataForChart.
  3. If there are valueLabels in the returned data, raw data values will be replaced with the labels via the function parseValueLabels.
  4. The returned data is pushed as a new object containing { title, data } to the this.state.charts array.
  5. Each of those objects is passed to the Chart.jsx disambiguation component.
  6. Depending on the chart type, a different component will be returned. So far, only EnrollmentBarChart.jsx has been programmed. That component will determine how data will be parsed and displayed.

If you are adding a chart type that is not bar, study-line, or category-line, it will need to be added to this if block in fetchDataForChart.

You will also need to create a new component along the lines of EnrollmentBarChart.jsx for any new chart type and add it to Chart.jsx.

Charts

Libraries

The main library used for the Enrollment Bar Chart is called Recharts. Recharts has very good documentation, so it's not necessary to expand much further here.

For generating an arbitrary number of distinct colors for the chart, we have used a small library called distinct-colors with these settings.

Data processing

Currently, fetchDataForChart returns data in the following format:

[
  {
    date: "2021-06-24",
    study: "FAKE_2",
    value: "White",
    varName: "chrdemo_racial_background",
  },
  ...
]

Note that the value field, which was originally a number, has been replaced with the label "White" by parseValueLabels.

Note also that date here refers to the consent date, because this is data for the Enrollment Bar Chart. But if the same data structure is reused for another chart type that does not deal strictly with enrollments, it could refer to the date of data collection or entry.

For the Enrollment Bar Chart, we process the data like this:

  1. Determine all unique values in the array for the value field and map each to a distinct color via the function getGroupings
  2. For each unique value in the array for the study field, count the total number of entries via the function getChartDataForStudy
  3. Also for each study, count the number of entries for each unique value within that study via the function getCounts
  4. Create an entry on the X-axis for each unique study
  5. For each entry on the X-axis, create a vertically-stacked bar for each unique value
  6. Display a label for each value with the percentage out of the total enrollment for that study

Size and spacing

You may want to make the sizing and margins of the chart dependent on the number of sites visualized rather than hard-coded.

Note also that one caveat to Recharts is that all of its positioning is relative, and it doesn't easily allow use of CSS helpers like Flexbox or Grid.

Clone this wiki locally