Known gaps and decisions still needed

This page is split by who needs the item. The first section is what bears on reading or citing the data: coverage limits, what a null means, which variables are thin, and where evidence is ambiguous. The second is what bears on building the database and the site, kept public so the decisions are not invisible, but not needed to use the data.

For researchers using the data

• Annual reports on the MCA platform are foundations only. A read-only probe of the report index (zwxt KBE242, 2026-07-29) asked for report-form types outside the collected set. Types 2, 7, 8, 9, and 10 return no rows, and an unfiltered page returns only 公募基金会年检 and 非公募基金会年检. No 社会团体 or 民办非企业单位 annual report exists in this index, so financial data for the other 1.19 million registered organizations is not obtainable from this current source. The canonical panel does publish evidence-mapped historical Beijing and Shanghai financial observations for some non-foundations, alongside their registry identity, but that coverage is regional, uneven, and not a substitute for a national annual-report feed. Under O2, non-foundations remain available in bulk data and registry search and do not receive thin longitudinal profile pages.
• Report-form types 0 and 1 are now collected. The default was 3,4,5,6, which omitted 12,307 foundation annual reports. RICF_CHARITY_REPORT_FORM_TYPES now defaults to 0,1,3,4,5,6, and the 2026-07-29 snapshot carries them: the filing index grew from 42,704 to 55,011 rows and the foundation-scoped index from 23,914 to 31,560 across 8,488 credit codes.
• Four indexed annual-report sections remain bounded negative results. Probes across KBE120–KBE300 found no annual-section relationship candidate for 履行信息公开义务情况 (31,521 rows), 监事意见 (31,521), 业务主管单位审查意见 (28,836), or 年度报告审核意见 (2,685). This is evidence about that tested resource range, not proof that no endpoint exists outside it. The KBE243–KBE300 sweep did find six further section-linked resources, all belonging to sections already harvested. Supervision, investment, organization-building, and other-explanation sections are collected, and their source-native fields now ship in releases; the fields remain source_native_unmapped until separately evidence-graded, which is a statement about their meaning rather than their availability.
• Profile fields now cover the whole registry, so a null means the source has no value. Registration authority, supervising unit, and registration date come from a per-organization endpoint with no batch form, and the crawl once ran type by type, which made a null ambiguous between “not collected yet” and “not in the source”. That is no longer the case: 1,198,369 of 1,198,950 published rows carry a profile row, and the 581 that do not are codes whose request the platform refused or reported absent. Eleven of the rows that do have a profile carry no value in any of the three fields. A null in one of these fields is therefore substantive. supervising_unit is null for 39,318 rows (3.3%) because an organization registered directly with the ministry has no separate supervising unit, and registered_date for 159,741 (13.3%). registry/manifest.json reports rows_with_profile_fields so the distinction stays checkable.
• The apparent 2025 fa_acco gap is a form change, not a collapse. Three organizations use the earlier aggregate activity-cost row. The other 5,451 use all six printed sublines—donation-project cost, service cost, goods cost, government-subsidy expenditure, taxes/surcharges, and other expenditure. No organization uses both representations, every subline filer has the complete six rows, and all 5,454 organizations are covered. The panel and overview use those sublines as an evidenced fallback; they are not relabelled as a filed aggregate.
• The consolidated RICF Codebook.xlsx documents every current processed field. Source-native akbe*, aaae*, and aae* fields are assigned meanings only where the platform or pipeline mapping verifies them; the remaining opaque fields are retained verbatim and marked source_native_unmapped rather than guessed.
• Two registry fields were established and are now published; seven are not. aaae0129 is the start-up fund and aaae0114 the registering authority’s division, both verified by identity test and published as start_up_fund and registration_division_code. aaae0106 pairs with the authority name across 3,730 codes, but 735 of them (19.7%) carry more than one name — mostly the same body written two ways, sometimes genuinely different bodies — so it is held back rather than published with a caveat a reader would not see. aaae0136, aaae0137, aaae0139, aaae0149, aae20having and aae21having are small enumerations and flags, populated for 96–100% of rows and skewed heavily to 0, with nothing in the codebook, the filing prints, or the platform’s code lists to say what they count. The platform resolves code lists by list name — C103, C106, ZB03, EX43 — not by field name, and the binding from field to list lives in its front end, so establishing these needs a live session rather than another pass over the data. They are named unknowns, not absences.
• No annual report has been collected for an organization that is not a foundation, and the report index does not serve one. Every annual-report table in the snapshot returns organization_type_code 3 only. The report index endpoint has been probed across form types 0–10: types 2 and 7–10 return no rows at all, and every row of the six that do return is a 公募基金会年检 or 非公募基金会年检 — a foundation annual inspection. The 55,011-row charitable index includes 6,487 organizations the registry types as 1 or 2, but those are a type disagreement between two source surfaces for the same organization, not association filings. If the platform publishes annual reports for other organization types, it is not through this endpoint.
• The current financial-statement tables are long-format and carry the official Chinese line-item label in the row. Every observed label in financial position, financial activities, and cash flow now resolves to a historical RICF variable, so those tables are no longer opaque. The four unlabeled financial-activities value columns were resolved by test rather than assumption: a cross-filing join identifies akbe1364/akbe1365 as the prior period (85.42% agreement; the strongest rejected alternative is 2.09%), the restricted-to-unrestricted transfer line identifies akbe1366 as unrestricted and akbe1367 as restricted (91.92% exact mirror), and the net asset change identity holds within each period’s columns independently (72.39% and 69.60%). The balance sheet’s akbe1349/akbe1350 resolve the same way (92.24%). Pass rates are recorded in the crosswalk and codebook.
• The historical fa_*s_* columns are undocumented in the original codebook. They satisfy s == e + u in 99.98% of non-zero 2016 rows, establishing them as the 合计 total, and are mapped on that basis.
• Six historical columns remain unmapped because the evidence is genuinely ambiguous: cf_inpc.1 (a duplicated column in the 2013 cash-flow table), GIS编码源地址 and 党员数 in 2015 (the latter could be either ba_pm or ba_bpm), and ba_evi, ba_seceml, and cfc_oid in 2016. They are listed in the crosswalk rather than resolved.
• GIS编码源地址 and GIS位置坐标 have no codebook equivalent, but their origin is documented: the 2016 geocoding scripts preserved under legacy/tooling/codes/Geocoding/ (mirrored as the standalone RICF_GIS repository) forward-geocoded each foundation’s registered address with ArcGIS, retried failures with Google, and wrote the geocoder-normalized address and [lat, lon] pair in exactly the format the 2015 basic profile carries. GIS编码源地址 is therefore the geocoder-normalized address — distinct from the filer’s self-reported address — and GIS位置坐标 the geocoded coordinate. The scripts’ own result files record a non-random failure cascade (3,344 addresses; 419, then 85, then 67 failures; 53 never geocoded), which is why the 2015 column is missing for some rows. The independent result files remain a reproducibility benchmark for those columns rather than a mapped source.
• The current KBE242 charity filing index covers 2019–2025. The imported historical layers cover earlier years unevenly. The absent 2018 filing feed is partly recoverable: Chinese annual reports restate the prior period alongside the current one, so 2019 filings supply 162,310 panel observations for 2018 across 2,753 foundations. These are labelled prior_period or opening_balance and are never presented as filed values. 2018 still has no filing of its own: the site counts it as data the database holds, marks it P in the coverage matrix rather than H or M, and dashes its row in the statement-completeness table because nothing was reported for it.
• The earliest filing an organization has also describes the year before it. A balance sheet’s opening balance is the same measurement as the previous year’s closing balance, so an organization whose first report is 2013 can carry a 2012 balance sheet. This was derived from current filings only, which is why 2018 was recoverable and 2012 was not; it now runs over every layer. 357,206 observations were added, most of them 2012, and the panel’s floor moves to 2004 — a year that exists only as recovered values, exactly as 2018 does. The identity is measured rather than assumed: where both are present, an opening balance equals the prior year’s closing balance in 96.88% of 498,301 pairs, and in 89.30% of the 145,350 that are non-zero. A recovered value is emitted only for an organization-year with no filing of its own; 1,289,097 candidates were suppressed because one exists.
• A recovered value stops at the year the organization was registered. An organization has no statement for a year it did not exist in, and the earliest filing’s opening balance describes the day before that filing’s year — which for a first filing is the day before the organization opened. 294,598 recovered observations are withheld on that basis, 97% of them exactly one year early. This was not only a consequence of deriving closing balances: prior-period columns had always been carried past the registration date, so discovery coverage falls from 67,062 organization-years to 64,261 and the derived-only count from 9,954 to 7,153. The test uses the earlier of the establishment and registration dates, so a value is dropped only where both agree the organization did not yet exist.
  • Filed values are kept even where they predate that date. 54,013 of them do, across 588 organizations and gaps of one to eleven years, which is the source disagreeing with itself often enough that deleting filed evidence on the strength of the date would destroy more than it protects.
  • 230 of the panel’s 16,712 organizations carry neither an establishment nor a registration date, so their recovered years cannot be tested at all. The 5,964 observations involved are published and counted in the panel summary as recovered_year_without_registration_date, because an untestable value is not a wrong one.
• The public detail layer links 22,400 historical organization-years from 2010–2017 and current evidence from 2019–2025. It contains 10,406 of 11,383 foundation records and 53,921 directly linked organization-years. Adding the reviewed panel projection, including archived HTML and neighbouring-filing evidence, raises discovery coverage to 10,431 foundations and 64,261 organization-years over 2004–2025; 7,153 are derived-only. Early archived years are sparse and do not imply national coverage.
• Historical source tables also contain 177 rows associated with 67 credit codes outside the canonical foundation index, including currently registered non-foundations. They remain in processed evidence, are excluded from foundation profile shards, and are counted in the detail manifest.
• The historical HTML extraction inventories 67,449 source records and parses 65,464 report documents with zero errors or encoding fallbacks. The selected chinanpo generations cover 11,698 of 11,742 organization-years; early coverage is sparse and jurisdictionally uneven. Only 594 form label/header contexts and 317 Shanghai table positions have graded mappings; other contexts remain source_native_unmapped. Stage 3f publishes only the privacy-safe financial projection—3,751,523 panel observations—and keeps the source-native long tables internal. Beijing 2016 and MOC 2013 independently agree with comparable panel values at 95.52%.
  • Because the source-native table stays internal, an archived-HTML year has no source row on the profile, and its statements are laid out from the panel instead. This is invisible to a reader by design: every year presents the same heading, the same sectioned statement and the same evidence block, whichever route its values took. 17,839 organization-year statements across roughly 2,500 profiles used to render as a flat list of every observation instead — every 2005–2012 year plus a tail of 2016–2017.
  • One limit remains on those years, and it is a limit of the data rather than of the presentation: only the three financial statements exist. The HTML mapping covers no basic-information or roster context, so those tabs are empty for 2005–2012 and say which years they do cover. A value there also carries no source-column tooltip, because the column is in the internal table; no substitute is shown in its place.
• Current annual_basic_information.parquet is published in full on the profile as one 基本信息 section, and every one of its 76 columns carries a codebook variable. 75 of the 76 labels are the printed heading of an official filing rendering or the result of a recorded identity test; the remaining 1 is published with no established meaning, because aaee0128 holds the value 1 in 176 of 31,521 filings and nothing names it. The forms of 2019, 2021, 2023 and 2025 differ, so the mapping cites a print for each rather than assuming one covers the others.
  • The section is no longer outside the harmonized panel. It reached the panel from the published legacy tables only, so it ran 2013–2017; it now runs 2013–2025 across 100 variables and 1,909,961 observations. 25 of those variables are counts or amounts and carry a number as well as the filed text, so 理事数 is a series rather than a string. A postcode or credit code is all digits and is deliberately not one of them.
  • What remains open is the other direction: 41 codebook basic variables have no counterpart in the current form at all — 年检情况, 评估情况, GIS位置坐标 and the like — so those series still stop in 2017. That is the source dropping the field, not a mapping gap.
• Personal contact details, including mobile numbers, are published in bulk. The withholding rule now covers dates of birth and nothing else. 3,437 personal mobile numbers that were previously removed are published, wherever a filer typed them — a dedicated mobile field, a landline box, or a name, website, address, or postcode field. So are the 1,609 person-attached and 728 organization email addresses whose local part is itself a mobile number, which were always published because the two cannot be separated. This is a maintainer decision on public-disclosure data rather than an oversight, and the reasoning is in the provenance policy. It is listed here because the consequence belongs with the other things a user should know before relying on this data: the published dataset is a bulk, downloadable index of contact routes for named individuals.
• The KBE183 full-time staff roster is now published, 2019–2025. 85,475 rows through the same privacy transform the board roster uses: current_staff_roster carries name, gender, whole-year age, political status, education, department and position, and the raw table stays internal because akbe1161 is a date of birth. 85,364 birth dates are withheld and 85,327 ages derived. Its labels come from the printed heading rows of two official filing renderings, because the form changed: 2019–2024 carry 性别 and one combined 所在部门及职务 column, and 2025 drops 性别, splits department from position, and adds an appointment period, an appointment form, and two flags for whether the organization pays the person’s social insurance and salary. Both shapes are published rather than merged, so a user can see which one their year used, and gender is null for every 2025 row because the form stopped asking. Five source columns remain unmapped: akbe1164, a party-organization name in 2019–2020 that varies between staff of the same organization and so is not the organization-level field the print carries; akbe1831/akbe1832, numeric on 64 of 85,475 rows; and two opaque platform identifiers.
• Current governance is now published for 2019–2025, but 2018 remains a gap. The privacy-safe KBE185 derivative contains 340,322 source rows: 276,566 board members and 63,756 supervisors. It retains filed names, positions, terms, compensation, gender, political status, and related public filing fields. It never publishes DOB; 340,312 DOB-bearing rows and 14 copied DOB values are counted as withheld, and 340,163 ages are derived in whole years at report-year end through src/ricf/privacy.py. Historical 2010–2017 rosters still use exact identity evidence rather than fuzzy matching, so unresolved rows remain counted instead of guessed.
• Duplicate panel evidence is ranked, not deleted. The retained panel has 828,347 duplicate observation keys / 1,875,071 rows: 594,380 cross-source and 233,967 within-table keys. Exactly one row per key has precedence_rank == 1; 828,171 keys agree, 76 differ only within the 0.011 rounding tolerance, and 6,303 substantive conflicts remain flagged. A further 1,525 rows in 718 identity-conflict keys are quarantined. Canonical filed analysis should filter observation_kind == "reported", precedence_rank == 1, and identity_conflict == false. Merged historical lineages outrank either single-source lineage; where chinanpo and CFC meet directly, chinanpo is primary. An independent 2016 prior-period comparison slightly favors CFC overall (97.6735% versus 97.4749% within one cent); that caveat and every displaced value remain explicit.

• On the conflicting keys specifically, the rank-1 choice is usually the one independent evidence contradicts. The comparison above is taken over all observations, where the two lineages agree with the arbiter almost equally. Conditioning on the cases where they disagree with each other reverses the picture. scripts/audit_duplicate_precedence.py re-derives it: of the 6,250 conflicting organization-year-variable groups, 1,109 have an observation of the same figure with a different observation_kind, and 326–373 of those (by rounding tolerance) are decidable once arbiters sharing a lineage with either candidate are excluded — 1,522 arbiter observations are excluded on that ground, because agreement with your own lineage is plumbing rather than evidence. On what remains, the rank-1 value is the one the independent observation backs in only 10.2–11.3% of cases: 5.9% for one_side_zero and 15.3–20.3% for non_zero_ratio, stable from a 0.01% to a 2% tolerance.

  This is a measurement, not a verdict, and the rule has not been changed on the strength of it. The arbiter is a third opinion rather than ground truth — panel_restatements.parquet exists because a later filing can legitimately disagree with a year’s own filing — and it reaches only about 5% of the conflicting keys. The obvious cheap rule does not survive contact either: only 56.1% of the 3,309 one_side_zero conflicts currently publish the zero, so “prefer the non-zero side” is not a description of what the rule is doing wrong. What the number does establish is that a user who takes the rank-1 value on a conflicting key is not getting a neutral coin flip, and until a wider arbitration source settles it, conflicting keys are better treated as unresolved than as resolved. Filter on values_disagree == false for analysis that cannot tolerate this.

• Widening that arbitration was attempted and both routes are closed. The measurement above reaches about 5% of the conflicting keys, so two ways of extending it were tested rather than assumed.

  A third source lineage cannot witness the dispute. The panel carries seven lineages, and beijing alone holds 1,492,235 reported observations. It does not help: panel_duplicate_observations.parquet contains every row for a duplicated key, so any lineage that observed that key is one of the candidates rather than an outside witness. This is a property of the key, not a gap in the data.

  An accounting identity is circular here. fp_ntt == fp_unls + fp_lims holds for 93.99% of unconflicted triples and covers 439 conflicting keys — more than the prior-period route — and taken at face value it contradicts the rank-1 choice in 99.3% of them. That number is an artefact. All three terms for one organization-year come from the same filing, and that filing is one of the disputed sources, so solving the identity reproduces a candidate’s own internal consistency. Every one of the 439 shares a lineage with a candidate and the independent count is exactly zero, which scripts/audit_duplicate_precedence.py records under accounting_identity_route so the trap stays visible instead of being rediscovered.

  The prior-period and opening-balance observations therefore remain the only independent arbiter available in this repository, and its coverage cannot currently be widened from within it.

• The canonical organization_panel.parquet retains 11,159,082 source observations across 16,712 organizations and 2004–2025. Its 1,302,542 rows before 2013 are marked sparse_early_year because Beijing and chinanpo are much denser than other jurisdictions. It contains 1,496,548 non-foundation observations, but public profile shards remain foundation-scoped under O2.
• The complete type-3 registry contains 10,956 valid foundation credit codes. Current profile lookup succeeds for 10,947; eight reproducibly return MCA application error err2, and one returns no profile. All nine remain in the status table.
• The registry has 459 valid credit codes whose prefixes do not match the usual 51/52/53 type convention. The explicit official registry type is retained as authoritative and validation reports the exceptions.
• organization_status_code and registry_organization_type_code values that the platform’s own code list defines now carry its published label on the public site. Observed values the list does not define keep the raw source code and receive no label.
• The 2013 supplemental index has 653 documents, all still pending human review. Filename-derived document kinds and provisional evidence grades are not factual attributions. Likewise, 92 recovered 2014 donation remarks remain internal pending free-text privacy review.

For database and site development

• The release gate records a measured ruling on organization-profile status errors. validate raises a status_errors: warning for any status table containing a non-ok row, and metadata/release_gate.json refuses a release on an unaccepted warning. The foundation profile crawl has such an accepted entry, justified by a small reproducible set of MCA application errors. That justification now has two snapshots behind it: 2026-07-28 and 2026-07-29 return the identical nine credit codes as non-ok, so the set is stable rather than intermittent. organization_profiles.status.parquet has now completed: 1,187,422 requests are ok, 551 are errors, and 21 are not found (572 non-ok of 1,187,994). Gate version 2 accepts that exact measured warning because every outcome remains in the request-status table and no registry entity is removed; a different or unreviewed warning still fails.
• The companion import intentionally skips a known truncated df_fin_position-20220804.pkl; live KBE194 collection replaces it.
• Public/protected decisions remain provisional. The generated Pages data applies conservative field-level exclusions, and a field-level review is required before expanding that publication set or deploying it broadly.
• The tracked legacy/nda.docx contains an email address. The maintainer explicitly chose to keep it unchanged and public; it remains outside the repository’s general software, data, and documentation license grants.
• The ignored 2026-07-28 development release predates embedded license metadata. It remains immutable; build the next distributable release under a new snapshot ID so the manifest and documentation payload carry the adopted licenses and attribution.