# Data Pipeline のデータ鮮度

Data Pipeline エクスポートのデータ可用性タイムラインについて説明します。

Data Pipeline は、定期的に Stripe データを宛先に送信します。データ到着までの時間はデータセットによって異なります。

> データの配信は顧客によって異なり、主にデータサイズによって決まります。登録後、実際の配信時間はデータウェアハウスで確認できます。

次の用語は、Data Pipeline のタイミングに関するものです。

- **頻度**: Stripe が送信先にデータを共有する頻度。たとえば、頻度が 3 時間の場合、データは 00:00、03:00、06:00 UTC などのタイミングで共有されます。
- **鮮度 P50 (時間)**: 50% の場合、イベント作成からこの期間内にデータを利用できます。
- **鮮度 P90 (時間)**: 90% の場合、イベント作成からこの期間内にデータを利用できます。
- **最大遅延 (時間)**: イベントの作成からデータが利用可能になるまでの最大予想時間。

たとえば、3 時間の頻度の場合、03:00 UTC バッチには 00:01 UTC に作成されたレコードが含まれます。通常、バッチが終了してから 3 時間後には、06:00 UTC までにデータが利用可能になります。つまり、イベント作成から P50 の鮮度は 6 時間です。

## パイプラインのデータ鮮度を理解する

Stripe ダッシュボードでは、データセットごとにデータのランディング時刻と「データ取得日時」を表示できます。すべてのタイムスタンプの定義は次のとおりです。

- **データが到着した時刻**: データが目的地に到着した最終時刻。
- **現在のデータ**: このタイムスタンプまでのすべての API バッキングデータが、エクスポートされたスナップショットに含まれます。これは、派生データセットによって異なります。
- **次回の同期開始時刻**: 次回データの読み込みが開始される時刻。

## Data Pipeline を使用したデータ読み込み時間のクエリ

Stripe はデータ共有の一環として `data_load_times` テーブルを提供しています。ウェアハウスでは、**LOADED (読み込み時間)** 列のデータを見ると、各テーブル内のデータが更新された日時を確認できます。
![](https://b.stripecdn.com/docs-statics-srv/assets/data_load_times.28ab89d873e1295653dcd2ce18629a1b.png)

上記の値をクエリで使用すると、各テーブルの最終更新日時を基準にしてデータをフィルタリングできます。次のクエリを使用すると、前週の取引残高に関するデータを取得できます。

```sql
select
  created,
  id,
  amount,
  currency,
  source_id
from
  balance_transactions,
  data_load_times
where
  data_load_times.table_name = 'balance_transactions'
  and datediff(day, created, data_load_times.loaded) <= 7
order by
  created desc
```

## データセットの鮮度

#### API ベースのデータ

| データセット        | テーブル名                                           | 頻度 (時間) | 鮮度 P50 (時間) | 鮮度 P90 (時間) | 最大遅延 (時間) |
| ------------- | ----------------------------------------------- | ------- | ----------- | ----------- | --------- |
| billing       | クーポン                                            | 3       | 6           | 7           | 12        |
| billing       | coupons_currency_options                        | 3       | 6           | 7           | 12        |
| billing       | coupons_metadata                                | 3       | 6           | 7           | 12        |
| billing       | credit_note_discount_amounts                    | 3       | 6           | 7           | 12        |
| billing       | credit_note_line_item_discount_amounts          | 3       | 6           | 7           | 12        |
| billing       | credit_note_line_item_tax_amounts               | 3       | 6           | 7           | 12        |
| billing       | credit_note_line_items                          | 3       | 6           | 7           | 12        |
| billing       | credit_note_tax_amounts                         | 3       | 6           | 7           | 12        |
| billing       | credit_notes                                    | 3       | 6           | 7           | 12        |
| billing       | credit_notes_metadata                           | 3       | 6           | 7           | 12        |
| billing       | discounts                                       | 3       | 6           | 7           | 12        |
| billing       | invoice_custom_fields                           | 3       | 6           | 7           | 12        |
| billing       | invoice_customer_tax_ids                        | 3       | 6           | 7           | 12        |
| billing       | invoice_items                                   | 3       | 6           | 7           | 12        |
| billing       | invoice_items_metadata                          | 3       | 6           | 7           | 12        |
| billing       | invoice_line_item_discount_amounts              | 3       | 6           | 7           | 12        |
| billing       | invoice_line_item_tax_amounts                   | 3       | 6           | 7           | 12        |
| billing       | invoice_line_items                              | 3       | 6           | 7           | 12        |
| billing       | invoice_payments                                | 3       | 6           | 7           | 12        |
| billing       | invoice_shipping_cost_taxes                     | 3       | 6           | 7           | 12        |
| billing       | invoices                                        | 3       | 6           | 7           | 12        |
| billing       | invoices_metadata                               | 3       | 6           | 7           | 12        |
| billing       | plans                                           | 3       | 6           | 7           | 12        |
| billing       | plans_metadata                                  | 3       | 6           | 7           | 12        |
| billing       | price_tiers                                     | 3       | 6           | 7           | 12        |
| billing       | prices                                          | 3       | 6           | 7           | 12        |
| billing       | prices_currency_options                         | 3       | 6           | 7           | 12        |
| billing       | prices_metadata                                 | 3       | 6           | 7           | 12        |
| billing       | products                                        | 3       | 6           | 7           | 12        |
| billing       | products_metadata                               | 3       | 6           | 7           | 12        |
| billing       | promotion_codes                                 | 3       | 6           | 7           | 12        |
| billing       | quotes                                          | 3       | 6           | 7           | 12        |
| billing       | subscription_items                              | 3       | 6           | 7           | 12        |
| billing       | subscription_items_metadata                     | 3       | 6           | 7           | 12        |
| billing       | subscription_schedule_phase_add_invoice_items   | 3       | 6           | 7           | 12        |
| billing       | subscription_schedule_phase_configuration_items | 3       | 6           | 7           | 12        |
| billing       | subscription_schedule_phases                    | 3       | 6           | 7           | 12        |
| billing       | subscription_schedule_phases_metadata           | 3       | 6           | 7           | 12        |
| billing       | subscription_schedules                          | 3       | 6           | 7           | 12        |
| billing       | subscription_schedules_metadata                 | 3       | 6           | 7           | 12        |
| billing       | subscriptions                                   | 3       | 6           | 7           | 12        |
| billing       | subscriptions_metadata                          | 3       | 6           | 7           | 12        |
| billing       | tax_rates                                       | 3       | 6           | 7           | 12        |
| billing       | tax_rates_metadata                              | 3       | 6           | 7           | 12        |
| billing       | usage_records                                   | 3       | 6           | 7           | 12        |
| capital       | financing_balances                              | 3       | 6           | 7           | 12        |
| capital       | financing_offers                                | 3       | 6           | 7           | 12        |
| checkout      | checkout_custom_fields                          | 3       | 6           | 7           | 12        |
| checkout      | checkout_line_items                             | 3       | 6           | 7           | 12        |
| checkout      | checkout_sessions                               | 3       | 6           | 7           | 12        |
| checkout      | payment_links                                   | 3       | 6           | 7           | 12        |
| connect       | accounts                                        | 3       | 6           | 7           | 12        |
| connect       | accounts_metadata                               | 3       | 6           | 7           | 12        |
| connect-fees  | application_fee_refunds                         | 3       | 6           | 7           | 12        |
| connect-fees  | application_fee_refunds_metadata                | 3       | 6           | 7           | 12        |
| connect-fees  | application_fees                                | 3       | 6           | 7           | 12        |
| crypto        | crypto_onramp_sessions                          | 3       | 6           | 7           | 12        |
| customers     | customer_balance_transactions                   | 3       | 6           | 7           | 12        |
| customers     | customer_balance_transactions_metadata          | 3       | 6           | 7           | 12        |
| customers     | customer_cash_balance_transactions              | 3       | 6           | 7           | 12        |
| customers     | customer_tax_ids                                | 3       | 6           | 7           | 12        |
| customers     | customers                                       | 3       | 6           | 7           | 12        |
| customers     | customers_metadata                              | 3       | 6           | 7           | 12        |
| issuing       | issuing_authorizations                          | 3       | 6           | 7           | 12        |
| issuing       | issuing_authorizations_metadata                 | 3       | 6           | 7           | 12        |
| issuing       | issuing_cardholders                             | 3       | 6           | 7           | 12        |
| issuing       | issuing_cardholders_metadata                    | 3       | 6           | 7           | 12        |
| issuing       | issuing_cards                                   | 3       | 6           | 7           | 12        |
| issuing       | issuing_cards_metadata                          | 3       | 6           | 7           | 12        |
| issuing       | issuing_disputes                                | 3       | 6           | 7           | 12        |
| issuing       | issuing_network_tokens                          | 3       | 6           | 7           | 12        |
| issuing       | issuing_transactions                            | 3       | 6           | 7           | 12        |
| issuing       | issuing_transactions_metadata                   | 3       | 6           | 7           | 12        |
| payments      | balance_transaction_fee_details                 | 3       | 6           | 7           | 12        |
| payments      | balance_transactions                            | 3       | 6           | 7           | 12        |
| payments      | charges                                         | 3       | 6           | 7           | 12        |
| payments      | charges_metadata                                | 3       | 6           | 7           | 12        |
| payments      | disputes                                        | 3       | 6           | 7           | 12        |
| payments      | disputes_enhanced_eligibility                   | 3       | 6           | 7           | 12        |
| payments      | disputes_metadata                               | 3       | 6           | 7           | 12        |
| payments      | payment_intents                                 | 3       | 6           | 7           | 12        |
| payments      | payment_intents_metadata                        | 3       | 6           | 7           | 12        |
| payments      | payment_method_details                          | 3       | 6           | 7           | 12        |
| payments      | payment_methods                                 | 3       | 6           | 7           | 12        |
| payments      | payment_methods_metadata                        | 3       | 6           | 7           | 12        |
| payments      | payment_reviews                                 | 3       | 6           | 7           | 12        |
| payments      | refunds                                         | 3       | 6           | 7           | 12        |
| payments      | refunds_metadata                                | 3       | 6           | 7           | 12        |
| payments      | rule_decisions                                  | 3       | 6           | 7           | 12        |
| payments      | setup_attempts                                  | 3       | 6           | 7           | 12        |
| payments      | setup_intents                                   | 3       | 6           | 7           | 12        |
| payments      | setup_intents_metadata                          | 3       | 6           | 7           | 12        |
| payments      | sources                                         | 3       | 6           | 7           | 12        |
| payments      | sources_metadata                                | 3       | 6           | 7           | 12        |
| tax-reporting | tax_forms                                       | 3       | 6           | 7           | 12        |
| terminal      | terminal_hardware_order_items                   | 3       | 6           | 7           | 12        |
| terminal      | terminal_hardware_order_metadata                | 3       | 6           | 7           | 12        |
| terminal      | terminal_hardware_order_shipment_tracking       | 3       | 6           | 7           | 12        |
| terminal      | terminal_hardware_order_tax_amounts             | 3       | 6           | 7           | 12        |
| terminal      | terminal_hardware_orders                        | 3       | 6           | 7           | 12        |
| transfers     | transfer_reversals                              | 3       | 6           | 7           | 12        |
| transfers     | transfer_reversals_metadata                     | 3       | 6           | 7           | 12        |
| transfers     | transfers                                       | 3       | 6           | 7           | 12        |
| transfers     | transfers_metadata                              | 3       | 6           | 7           | 12        |
| treasury      | treasury_financial_accounts                     | 3       | 6           | 7           | 12        |
| treasury      | treasury_financial_accounts_metadata            | 3       | 6           | 7           | 12        |
| treasury      | treasury_inbound_transfers                      | 3       | 6           | 7           | 12        |
| treasury      | treasury_inbound_transfers_metadata             | 3       | 6           | 7           | 12        |
| treasury      | treasury_outbound_payments                      | 3       | 6           | 7           | 12        |
| treasury      | treasury_outbound_payments_metadata             | 3       | 6           | 7           | 12        |
| treasury      | treasury_outbound_transfers                     | 3       | 6           | 7           | 12        |
| treasury      | treasury_outbound_transfers_metadata            | 3       | 6           | 7           | 12        |
| treasury      | treasury_transaction_entries                    | 3       | 6           | 7           | 12        |
| treasury      | treasury_transactions                           | 3       | 6           | 7           | 12        |

#### 派生データ

| データセット   | テーブル名                                                               | 頻度 (時間) | 鮮度 P50 (時間) | 鮮度 P90 (時間) | 最大遅延 (時間) |
| -------- | ------------------------------------------------------------------- | ------- | ----------- | ----------- | --------- |
| billing  | usage_based_billing_billing_alerts                                  | 3       | 5           | 6           | 15        |
| billing  | usage_based_billing_billing_alert_filters                           | 3       | 5           | 6           | 15        |
| billing  | usage_based_billing_billing_alert_metadata                          | 3       | 5           | 6           | 15        |
| billing  | customer_change_events                                              | 24      | 41          | 43          | 79        |
| billing  | billing_cadences_beta                                               | 3       | 5           | 6           | 15        |
| billing  | billing_credit_balance_transactions                                 | 3       | 7           | 7           | 15        |
| billing  | billing_credit_grant_metadata                                       | 3       | 6           | 7           | 15        |
| billing  | billing_credit_grants                                               | 3       | 6           | 7           | 15        |
| billing  | billing_intents_beta                                                | 3       | 5           | 6           | 15        |
| billing  | billing_meters                                                      | 6       | 10          | 11          | 19        |
| billing  | billing_meter_dimensions                                            | 6       | 10          | 11          | 19        |
| billing  | billing_meter_event_summaries                                       | 6       | 10          | 11          | 19        |
| billing  | billing_meter_event_summary_segments                                | 6       | 10          | 11          | 19        |
| billing  | billing_meter_invalid_events                                        | 6       | 10          | 11          | 19        |
| billing  | billing_meter_invalid_events_payload                                | 6       | 10          | 11          | 19        |
| billing  | subscription_item_change_events_v2_beta                             | 3       | 6           | 7           | 15        |
| billing  | license_fee_versions_beta                                           | 3       | 5           | 6           | 15        |
| billing  | license_fees_beta                                                   | 3       | 5           | 6           | 15        |
| billing  | licensed_items_beta                                                 | 3       | 5           | 6           | 15        |
| billing  | metered_items_beta                                                  | 3       | 5           | 6           | 15        |
| billing  | pricing_plan_components_beta                                        | 3       | 5           | 6           | 15        |
| billing  | pricing_plan_subscriptions_beta                                     | 3       | 5           | 6           | 15        |
| billing  | pricing_plan_versions_beta                                          | 3       | 5           | 6           | 15        |
| billing  | pricing_plans_beta                                                  | 3       | 5           | 6           | 15        |
| billing  | rate_card_rates_beta                                                | 3       | 5           | 6           | 15        |
| billing  | rate_card_subscriptions_beta                                        | 3       | 5           | 6           | 15        |
| billing  | rate_card_versions_beta                                             | 3       | 5           | 6           | 15        |
| billing  | rate_cards_beta                                                     | 3       | 5           | 6           | 15        |
| billing  | recoveries                                                          | 24      | 45          | 50          | 79        |
| billing  | subscription_item_change_events                                     | 24      | 42          | 45          | 55        |
| billing  | service_actions_beta                                                | 3       | 5           | 6           | 15        |
| billing  | subscriptions_paid_usage                                            | 3       | 7           | 8           | 18        |
| connect  | v2__accounts                                                        | 24      | 34          | 38          | 55        |
| connect  | v2__accounts_metadata                                               | 24      | 34          | 38          | 55        |
| cost     | network_cost_insights_report                                        | 24      | 59          | 85          | 151       |
| other    | acceptance_reporting_v3_itemized                                    | 24      | 46          | 58          | 99        |
| other    | activity_lineitems                                                  | 24      | 83          | 93          | 130       |
| other    | connected_account_activity_lineitems                                | 24      | 83          | 93          | 130       |
| other    | analytics_page_events                                               | 24      | 38          | 44          | 103       |
| other    | iins                                                                | 24      | 29          | 30          | 55        |
| other    | cau_fees                                                            | 24      | 64          | 72          | 103       |
| other    | charge_groups                                                       | 24      | 43          | 48          | 103       |
| other    | connect_card_payments_fees_plan_level                               | 24      | 54          | 58          | 103       |
| other    | connect_card_payments_fees_transaction_level                        | 24      | 54          | 58          | 103       |
| other    | connected_account_itemized_fees                                     | 24      | 44          | 48          | 151       |
| other    | connected_account_itemized_fees_beta                                | 24      | 43          | 47          | 151       |
| other    | connected_account_summarized_balance_transactions                   | 6       | 12          | 12          | 23        |
| other    | disputes_reporting_v1_itemized                                      | 24      | 41          | 55          | 99        |
| other    | disputes_reporting_v1_itemized_v2                                   | 24      | 69          | 84          | 99        |
| other    | exchange_rates_from_usd                                             | 24      | 29          | 30          | 55        |
| other    | fee_credits_activities                                              | 6       | 10          | 11          | 21        |
| other    | icplus_fees                                                         | 24      | 56          | 60          | 103       |
| other    | itemized_fees                                                       | 24      | 44          | 48          | 151       |
| other    | itemized_fees_beta                                                  | 24      | 43          | 47          | 151       |
| other    | revenue_recognition_debits_and_credits                              | 6       | 9           | 9           | 16        |
| other    | revenue_recognition_exclusions                                      | 6       | 9           | 10          | 16        |
| other    | revenue_recognition_manual_journal_entries                          | 24      | 31          | 32          | 55        |
| other    | summarized_balance_transactions                                     | 6       | 12          | 12          | 23        |
| other    | visa_cedp_report                                                    | 24      | 26          | 29          | 151       |
| payments | payins_insights_lightning_astro_deduped_aggregated_with_attempts_v2 | 24      | 46          | 54          | 79        |
| payments | payins_insights_lightning_astro_raw_aggregated_with_attempts_v2     | 24      | 36          | 42          | 79        |
| payments | v2_us_bank_accounts                                                 | 24      | 29          | 31          | 55        |
| radar    | radar_data_integration                                              | 24      | 40          | 45          | 79        |
| radar    | radar_rule_attributes                                               | 24      | 38          | 42          | 79        |
| radar    | radar_rules                                                         | 24      | 34          | 37          | 55        |
| tax      | tax_transaction_customer_tax_ids                                    | 24      | 35          | 39          | 55        |
| tax      | tax_transaction_jurisdiction_details                                | 24      | 35          | 39          | 55        |
| tax      | tax_transaction_line_items                                          | 24      | 35          | 39          | 55        |
| tax      | tax_transaction_line_items_metadata                                 | 24      | 35          | 39          | 55        |
| tax      | tax_transaction_shipping_costs                                      | 24      | 35          | 39          | 55        |
| tax      | tax_transactions                                                    | 24      | 35          | 39          | 55        |
| tax      | tax_transactions_metadata                                           | 24      | 35          | 39          | 55        |
| terminal | terminal_locations                                                  | 24      | 27          | 27          | 151       |
| terminal | terminal_readers                                                    | 24      | 27          | 27          | 151       |
| treasury | v2__adjustments                                                     | 3       | 5           | 8           | 22        |
| treasury | v2__inbound_transfers                                               | 3       | 5           | 8           | 22        |
| treasury | v2__inbound_transfers_history                                       | 3       | 5           | 8           | 22        |
| treasury | v2__outbound_payments                                               | 3       | 5           | 8           | 22        |
| treasury | v2__outbound_payments_metadata                                      | 3       | 5           | 8           | 22        |
| treasury | v2__outbound_transfers                                              | 3       | 5           | 8           | 22        |
| treasury | v2__received_credits                                                | 3       | 5           | 8           | 22        |
| treasury | v2__transaction_entries                                             | 3       | 5           | 8           | 22        |
| treasury | v2__transactions                                                    | 3       | 5           | 8           | 22        |
| treasury | v2__us_bank_accounts                                                | 3       | 5           | 8           | 22        |