Skip to content

Update flusurv schema and docs with new age, sex, and race groups #1287

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
May 8, 2024
Merged

Update flusurv schema and docs with new age, sex, and race groups #1287

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
ed4dcf9
add new columns to flusurv endpoint spec
nmdefries Sep 15, 2023
73901c6
add new age, sex, race strata, and season to schema
nmdefries Sep 15, 2023
108a000
list new signals in documentation
nmdefries Sep 15, 2023
16dc0c3
move new season column to end of table
nmdefries Sep 18, 2023
9a481de
add migration script to add new fields to flusurv table
nmdefries Sep 18, 2023
3e5f5b6
add field descriptions; reformat table so don't need to match line le…
nmdefries Sep 27, 2023
cd9a46b
point from acquisition script to docs and schema def to deduplicate
nmdefries Sep 27, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
50 changes: 34 additions & 16 deletions docs/api/flusurv.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -52,22 +52,40 @@ If neither is specified, the current issues are used.

## Response

| Field | Description | Type |
|--------------------------|-----------------------------------------------------------------|------------------|
| `result` | result code: 1 = success, 2 = too many results, -2 = no results | integer |
| `epidata` | list of results | array of objects |
| `epidata[].release_date` | | string |
| `epidata[].location` | | string |
| `epidata[].issue` | | integer |
| `epidata[].epiweek` | | integer |
| `epidata[].lag` | | integer |
| `epidata[].rate_age_0` | | float |
| `epidata[].rate_age_1` | | float |
| `epidata[].rate_age_2` | | float |
| `epidata[].rate_age_3` | | float |
| `epidata[].rate_age_4` | | float |
| `epidata[].rate_overall` | | float |
| `message` | `success` or error message | string |
| Field | Description | Type |
|---|---|---|
| `result` | result code: 1 = success, 2 = too many results, -2 = no results | integer |
| `epidata` | list of results | array of objects |
| `epidata[].release_date` | the date when this record was first published by the CDC | string |
| `epidata[].location` | the name of the catchment (e.g. 'network_all', 'CA', 'NY_albany' | string |
| `epidata[].issue` | the epiweek of publication (e.g. issue 201453 includes epiweeks up to and including 2014w53, but not 2015w01 or following) | integer |
| `epidata[].epiweek` | the epiweek during which the data was collected | integer |
| `epidata[].lag` | number of weeks between `epiweek` and `issue` | integer |
| `epidata[].rate_age_0` | hospitalization rate for ages 0-4 | float |
| `epidata[].rate_age_1` | hospitalization rate for ages 5-17 | float |
| `epidata[].rate_age_2` | hospitalization rate for ages 18-49 | float |
| `epidata[].rate_age_3` | hospitalization rate for ages 50-64 | float |
| `epidata[].rate_age_4` | hospitalization rate for ages 65+ | float |
| `epidata[].rate_overall` | overall hospitalization rate | float |
| `epidata[].rate_age_5` | hospitalization rate for ages 65-74 | float |
| `epidata[].rate_age_6` | hospitalization rate for ages 75-84 | float |
| `epidata[].rate_age_7` | hospitalization rate for ages 85+ | float |
| `epidata[].rate_age_18t29` | hospitalization rate for ages 18 to 29 | float |
| `epidata[].rate_age_30t39` | hospitalization rate for ages 30 to 39 | float |
| `epidata[].rate_age_40t49` | hospitalization rate for ages 40 to 49 | float |
| `epidata[].rate_age_5t11` | hospitalization rate for ages 5 to 11 | float |
| `epidata[].rate_age_12t17` | hospitalization rate for ages 12 to 17 | float |
| `epidata[].rate_age_lt18` | hospitalization rate for ages <18 | float |
| `epidata[].rate_age_gte18` | hospitalization rate for ages >=18 | float |
| `epidata[].rate_race_white` | hospitalization rate for white people | float |
| `epidata[].rate_race_black` | hospitalization rate for black people | float |
| `epidata[].rate_race_hisp` | hospitalization rate for Hispanic/Latino people | float |
| `epidata[].rate_race_asian` | hospitalization rate for Asian people | float |
| `epidata[].rate_race_natamer` | hospitalization rate for American Indian/Alaskan Native people | float |
| `epidata[].rate_sex_male` | hospitalization rate for males | float |
| `epidata[].rate_sex_female` | hospitalization rate for females | float |
| `epidata[].season` | indicates the start and end years of the winter flu season in the format YYYY-YY (e.g. 2022-23 indicates the flu season running late 2022 through early 2023) | string |
| `message` | `success` or error message | string |

Notes:
* The `flusurv` age groups are, in general, not the same as the ILINet
Expand Down
39 changes: 3 additions & 36 deletions src/acquisition/flusurv/flusurv_update.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,42 +19,9 @@
=== Data Dictionary ===
=======================

`flusurv` is the table where US flu hospitalization rates are stored.
+--------------+-------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+--------------+-------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| release_date | date | NO | MUL | NULL | |
| issue | int(11) | NO | MUL | NULL | |
| epiweek | int(11) | NO | MUL | NULL | |
| location | varchar(32) | NO | MUL | NULL | |
| lag | int(11) | NO | MUL | NULL | |
| rate_age_0 | double | YES | | NULL | |
| rate_age_1 | double | YES | | NULL | |
| rate_age_2 | double | YES | | NULL | |
| rate_age_3 | double | YES | | NULL | |
| rate_age_4 | double | YES | | NULL | |
| rate_overall | double | YES | | NULL | |
| rate_age_5 | double | YES | | NULL | |
| rate_age_6 | double | YES | | NULL | |
| rate_age_7 | double | YES | | NULL | |
+--------------+-------------+------+-----+---------+----------------+
id: unique identifier for each record
release_date: the date when this record was first published by the CDC
issue: the epiweek of publication (e.g. issue 201453 includes epiweeks up to
and including 2014w53, but not 2015w01 or following)
epiweek: the epiweek during which the data was collected
location: the name of the catchment (e.g. 'network_all', 'CA', 'NY_albany')
lag: number of weeks between `epiweek` and `issue`
rate_age_0: hospitalization rate for ages 0-4
rate_age_1: hospitalization rate for ages 5-17
rate_age_2: hospitalization rate for ages 18-49
rate_age_3: hospitalization rate for ages 50-64
rate_age_4: hospitalization rate for ages 65+
rate_overall: overall hospitalization rate
rate_age_5: hospitalization rate for ages 65-74
rate_age_6: hospitalization rate for ages 75-84
rate_age_7: hospitalization rate for ages 85+
`flusurv` is the table where US flu hospitalization rates are stored. See
`strc/ddl/fluview.sql` for the `flusurv` schema. See `docs/api/flusurv.md` for
field descriptions.

=================
=== Changelog ===
Expand Down
70 changes: 50 additions & 20 deletions src/ddl/fluview.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -329,7 +329,7 @@ CREATE TABLE `fluview_public` (
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

/*
`flusurv` stores FluSurv-NET data (flu hospitaliation rates) as published by
`flusurv` stores FluSurv-NET data (flu hospitalization rates) as published by
CDC.

Data is public.
Expand All @@ -345,25 +345,40 @@ Note that the flusurv age groups are, in general, not the same as the ILINet
particular "catchment" (e.g. 'network_all', 'CA', 'NY_albany') rather than by
regions and states in general.

+--------------+-------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+--------------+-------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| release_date | date | NO | MUL | NULL | |
| issue | int(11) | NO | MUL | NULL | |
| epiweek | int(11) | NO | MUL | NULL | |
| location | varchar(32) | NO | MUL | NULL | |
| lag | int(11) | NO | MUL | NULL | |
| rate_age_0 | double | YES | | NULL | |
| rate_age_1 | double | YES | | NULL | |
| rate_age_2 | double | YES | | NULL | |
| rate_age_3 | double | YES | | NULL | |
| rate_age_4 | double | YES | | NULL | |
| rate_overall | double | YES | | NULL | |
| rate_age_5 | double | YES | | NULL | |
| rate_age_6 | double | YES | | NULL | |
| rate_age_7 | double | YES | | NULL | |
+--------------+-------------+------+-----+---------+----------------+
+-------------------+-------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-------------------+-------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| release_date | date | NO | MUL | NULL | |
| issue | int(11) | NO | MUL | NULL | |
| epiweek | int(11) | NO | MUL | NULL | |
| location | varchar(32) | NO | MUL | NULL | |
| lag | int(11) | NO | MUL | NULL | |
| rate_age_0 | double | YES | | NULL | |
| rate_age_1 | double | YES | | NULL | |
| rate_age_2 | double | YES | | NULL | |
| rate_age_3 | double | YES | | NULL | |
| rate_age_4 | double | YES | | NULL | |
| rate_overall | double | YES | | NULL | |
| rate_age_5 | double | YES | | NULL | |
| rate_age_6 | double | YES | | NULL | |
| rate_age_7 | double | YES | | NULL | |
| rate_age_18t29 | double | YES | | NULL | |
| rate_age_30t39 | double | YES | | NULL | |
| rate_age_40t49 | double | YES | | NULL | |
| rate_age_5t11 | double | YES | | NULL | |
| rate_age_12t17 | double | YES | | NULL | |
| rate_age_lt18 | double | YES | | NULL | |
| rate_age_gte18 | double | YES | | NULL | |
| rate_race_white | double | YES | | NULL | |
| rate_race_black | double | YES | | NULL | |
| rate_race_hisp | double | YES | | NULL | |
| rate_race_asian | double | YES | | NULL | |
| rate_race_natamer | double | YES | | NULL | |
| rate_sex_male | double | YES | | NULL | |
| rate_sex_female | double | YES | | NULL | |
| season | char(7) | YES | | NULL | |
+-------------------+-------------+------+-----+---------+----------------+
*/

CREATE TABLE `flusurv` (
Expand All @@ -382,6 +397,21 @@ CREATE TABLE `flusurv` (
`rate_age_5` double DEFAULT NULL,
`rate_age_6` double DEFAULT NULL,
`rate_age_7` double DEFAULT NULL,
`rate_age_18t29` double DEFAULT NULL,
`rate_age_30t39` double DEFAULT NULL,
`rate_age_40t49` double DEFAULT NULL,
`rate_age_5t11` double DEFAULT NULL,
`rate_age_12t17` double DEFAULT NULL,
`rate_age_lt18` double DEFAULT NULL,
`rate_age_gte18` double DEFAULT NULL,
`rate_race_white` double DEFAULT NULL,
`rate_race_black` double DEFAULT NULL,
`rate_race_hisp` double DEFAULT NULL,
`rate_race_asian` double DEFAULT NULL,
`rate_race_natamer` double DEFAULT NULL,
`rate_sex_male` double DEFAULT NULL,
`rate_sex_female` double DEFAULT NULL,
`season` char(7) DEFAULT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `issue` (`issue`,`epiweek`,`location`),
KEY `release_date` (`release_date`),
Expand Down
18 changes: 18 additions & 0 deletions src/ddl/migrations/flusurv_age_sex_race_strata.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
-- Add new age, race, and sex strata, and season descriptor (YYYY-YY format)
ALTER TABLE `flusurv` ADD (
`rate_age_18t29` double DEFAULT NULL,
`rate_age_30t39` double DEFAULT NULL,
`rate_age_40t49` double DEFAULT NULL,
`rate_age_5t11` double DEFAULT NULL,
`rate_age_12t17` double DEFAULT NULL,
`rate_age_lt18` double DEFAULT NULL,
`rate_age_gte18` double DEFAULT NULL,
`rate_race_white` double DEFAULT NULL,
`rate_race_black` double DEFAULT NULL,
`rate_race_hisp` double DEFAULT NULL,
`rate_race_asian` double DEFAULT NULL,
`rate_race_natamer` double DEFAULT NULL,
`rate_sex_male` double DEFAULT NULL,
`rate_sex_female` double DEFAULT NULL,
`season` char(7) DEFAULT NULL,
);
23 changes: 22 additions & 1 deletion src/server/endpoints/flusurv.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ def handle():
# basic query info
q = QueryBuilder("flusurv", "fs")

fields_string = ["release_date", "location"]
fields_string = ["release_date", "location", "season"]
fields_int = ["issue", "epiweek", "lag"]
fields_float = [
"rate_age_0",
Expand All @@ -28,6 +28,27 @@ def handle():
"rate_age_3",
"rate_age_4",
"rate_overall",

"rate_age_5",
"rate_age_6",
"rate_age_7",

"rate_age_18t29",
"rate_age_30t39",
"rate_age_40t49",
"rate_age_5t11",
"rate_age_12t17",
"rate_age_lt18",
"rate_age_gte18",

"rate_race_white",
"rate_race_black",
"rate_race_hisp",
"rate_race_asian",
"rate_race_natamer",

"rate_sex_male",
"rate_sex_female",
]
q.set_fields(fields_string, fields_int, fields_float)
q.set_sort_order("epiweek", "location", "issue")
Expand Down