Data Collection

Data collection turns indicator and breakdown definitions into stored scores and snapshots.

Data collection jobs

A data collection job (All > Platform Analytics Administration > Data Collector > Jobs) collects scores for the automated indicators in its Indicators related list (its job indicators). Its Run field sets when it executes (daily, weekly, monthly, periodically, once, or on demand); the period length of the scores comes from the indicator Frequency, so the job schedule is set to match it. Each solution provides its own jobs, typically one historic job and one periodic job, such as [PA Incident] Historic Data Collection and [PA Incident] Daily Data Collection, and solution periodic jobs must be activated. Each run produces:

• Scores: one per indicator per period, plus one per breakdown element (and matrix combination) where breakdowns apply. Scores are stored in Scores Level 1 [pa_scores_l1] and Scores Level 2 [pa_scores_l2]; the old Scores [pa_scores] table is deprecated.
• Snapshots: the lists of record sys_ids collected with the scores, stored in Snapshot [pa_snapshots]. Only automated indicators with Collect records enabled have snapshots. They enable drilling down from a score to the records behind it, as they were at collection time.

Historical data collection

Run a historical data collection job once to collect scores and snapshots for existing records, for example when you set up a new indicator. Key rules:

• Set its Run field to On demand; never run historical jobs on a fixed schedule.
• A historical job deletes any previously collected data for the periods and indicators it covers (and nothing outside that range or those indicators), so avoid running it more than once and consider a separate job for a new indicator.
• It reconstructs history from current record values, so exclude breakdowns on fields whose values change over time (such as Assignment group or State), because their historical results would be meaningless.
• To add history for a newly added breakdown without overwriting indicator scores, use a one-off job limited to that breakdown with Collect indicator cleared.

Operating and troubleshooting collection

• Job logs (Data Collector > Job Logs) show each run's State, Run time, warnings, and errors, and the job log details describe each step per indicator source and collection period. They are the first place to look for missing scores.
• Diagnostics (Troubleshooting > Diagnostics) run predefined scripts that find configuration issues.
• Common causes of missing scores: the job did not run or the indicator is not in it, the source or indicator conditions exclude the records, the breakdown is not assigned or mapped, or the breakdown source exceeds the element limit.
• Retention: com.snc.pa.dc.keep_scores_for.frequency sets how many periods of scores are kept (732 for daily indicators by default), and snapshots are kept for 183 days by default (com.snc.pa.dc.keep_snapshots_for.frequency); the scheduled cleanup job deletes older data. Both can be overridden per indicator.
• Value when nil fills a period with a chosen score when no value is collected.

Chain of cause and effect: indicator source defines records → data collection job runs → produces scores (time series) and, with Collect records, snapshots → scores feed visualizations, KPI Details and the Analytics Hub → snapshots enable drilldown.