← All articles

Citable Research Platforms: What an RFP Should Require

What an RFP should require to operationalize a validated psychometric instrument without eroding it. Eight items that protect the instrument, the data, and the field.

Bert Carroll ·
LinkedIn RSS

A reference for instrument authors and program leads writing an RFP to operationalize a validated psychometric or community measurement instrument. The requirements below distinguish a research-grade platform from a survey form with a logo on it. They protect the instrument, the data, and the field that depends on both.

Why This Document Exists

Validated psychometric instruments are valuable and fragile. Years of development, validation, normative-sample work, and field testing produce something the research community can trust. A digital platform built carelessly can erase that trust in a single deployment cycle.

The most common failure mode is well-meaning. A funding organization issues an RFP for a “data platform” or “online assessment tool.” The RFP describes user-facing features but does not specify what makes the platform research-grade. Bids come in. The lowest bid is from a survey-form vendor with no understanding of psychometric integrity, longitudinal comparability, or research data publication. The vendor wins on price, deploys a brochure with a form, and the instrument’s validity claim quietly erodes.

This document describes what the RFP should require to prevent that outcome. It is written for instrument authors and program leads who want to protect the instrument they are putting into the world.

The pattern this is meant to prevent:

  • A survey-form vendor wins on price.
  • The platform deploys, looks fine, and collects responses.
  • Items get reworded for “user experience.” Scoring becomes “configurable” so administrators can tune it.
  • The dataset has no DOI, no version control, no data dictionary.
  • Two years in, no peer-reviewed paper can cite the data. The instrument’s validity claim no longer applies because the items have drifted.
  • The field loses the tool. The funder loses the investment.

Requirement 1: Instrument Fidelity Is Inviolable

The single most important requirement, and the one most often missing from non-specialist RFPs: the instrument cannot be modified through the platform.

Items, response scale, scoring formulas, domain assignments, phase classification thresholds, and recommendation copy are not configurable platform features. They are the instrument. A vendor who edits items, reorders questions, or “tunes” scoring breaks the instrument’s validity claim immediately.1 Any feature that exposes these elements to user-side modification is a fail condition, no matter how it is labeled.

The RFP should require:

  • The vendor will not modify items, response scale, scoring, domain assignments, phase thresholds, or recommendation copy through the platform interface, by anyone, ever.
  • Updates to the instrument occur only as versioned releases issued by the instrument author. The platform serves the new version alongside prior versions so historical responses remain interpretable.
  • Every response record identifies the instrument version that produced it.
  • The platform supports parallel instrument variants such as language translations, youth or short forms, and culturally adapted versions, served as separate authorized variants issued by the instrument author. Variants are served alongside the primary instrument; they are not user-configurable adaptations.
  • The platform architecturally separates the instrument layer (immutable) from the analysis layer (configurable). Cohort grouping, comparison sets, dashboards, exports, and reporting templates are configurable. The instrument is not.

A vendor who proposes a “configurable scoring engine” or an “admin tool to update items when authorized” has misunderstood the project. The RFP should make clear that this is not a feature; it is a destruction mechanism.

Requirement 2: Citability

A platform that produces uncitable data is a survey tool, not research infrastructure. The line between the two is decided by six attributes that, taken together, let a researcher reference the dataset in a peer-reviewed publication.

The same attributes align the platform with the FAIR principles for research data: Findable, Accessible, Interoperable, Reusable. FAIR is the de facto standard funders and journals use to evaluate research data infrastructure.2

The RFP should require:

  1. Persistent identifier (DOI). Every published dataset snapshot receives a Digital Object Identifier issued through a registered service such as Zenodo or DataCite.3 Vendor names the service. The DOI persists across platform redesigns, hosting migrations, and ownership changes.
  2. Versioned dataset snapshots. Snapshots generated on a defined cadence, annual at minimum, plus on-demand for publication cycles or reporting milestones. Each snapshot carries a snapshot date, a version number, and immutable contents.
  3. Stable URLs. Snapshot URLs do not change for the operational life of the platform, independent of underlying storage migration. A citation written today resolves correctly five years from now.
  4. Published data dictionary. Variable definitions, response coding, scoring rules, instrument version, and language variant for every field in every snapshot. Published as both human-readable PDF and machine-readable metadata in a recognized standard such as DDI-Codebook (DDI-XML)4 or JSON Schema with documented metadata headers. Generic CSV with no metadata is not acceptable.
  5. Documented collection methodology. Sampling description, instrument administration mode, consent posture, known limitations, and de-identification controls applied. The methodology document is published alongside each snapshot and updated when methodology changes.
  6. Pre-formatted suggested citation. APA and Chicago citation strings generated for each snapshot, copy-paste ready for inclusion in publications.

A vendor who answers requirement 2 with a generic “yes, we can provide data exports” has not understood the question. A serious response names the DOI registration service, describes the snapshot cadence, and shows an example data dictionary from prior work or a credible plan to produce one.

Requirement 3: Public, Organizational, and Administrative Modes Are Different Products

Anonymous public assessment, authenticated organizational use, and administrative oversight serve different participants under different consent and data-handling postures. A platform that conflates them collects personally identifiable information when it should not, or fails to support the program use case when it should.

The RFP should require:

  • An anonymous public mode with no login, no personally identifying information collected, and aggregate-only storage with de-identification controls (k-anonymity threshold,5 suppression of small cells in public reports).
  • An authenticated organizational mode with consent-based data collection, longitudinal tracking at defined intervals (baseline, follow-up, exit), and program-level dashboards.
  • Single sign-on via SAML 2.0 or OpenID Connect available in the authenticated mode, since deploying organizations such as universities, hospital systems, and government agencies typically require federation with their existing identity provider.
  • An administrative tier for system configuration, account creation, role assignment, data oversight, and dataset publication.
  • Multi-tier account hierarchy that supports individual, program, community, and administrative scopes with role-based permissions per tier.
  • Data isolation enforced at the database layer (row-level security or equivalent server-side enforcement). Application-layer filtering is not acceptable; it is a single misconfigured query away from cross-organization data exposure.

Requirement 4: Operational Posture for PII

Even when no protected health information is collected, platforms that handle personally identifiable information for participants in research studies require a defensible operational posture. This is not optional under the procurement standards of municipal, state, or federally-funded work.

The RFP should require:

  • Hosting in a region appropriate to the deploying jurisdiction (US-region for US-funded programs; Canadian, EU, UK, or other regions when the deploying organization, funder, or applicable data-protection law requires it), with documented disaster recovery objectives (RPO and RTO).
  • Encryption in transit (TLS 1.2 or higher) and at rest (AES-256 or equivalent).
  • Multi-factor authentication on all administrative accounts.
  • Audit logging covering user access, administrative actions, exports, uploads, and data changes, with multi-year retention.
  • Documented data retention and deletion policies.
  • Incident response and breach notification procedures.
  • WCAG 2.2 AA accessibility (2.1 AA at minimum)6 and responsive design across desktop, tablet, and mobile. Public-facing assessment surfaces especially must be usable by participants with disabilities and on the device they actually have.
  • An upgrade path to handling protected health information under a Business Associate Agreement when a deploying organization is a HIPAA covered entity.7

Requirement 5: Federal Grant-Reporting Compatibility

If the platform supports a federally-funded program, the RFP should name the grant-reporting frameworks the platform must support and require the vendor to describe how exports map to those frameworks.

The RFP should require:

  • Vendor cites the specific frameworks by name. For example, SAMHSA Mental Health Block Grant (MHBG), ReCAST cross-site indicators, SPARS,8 or NIH and HRSA equivalents, rather than offering generic “grant reporting” capability.
  • Vendor describes the export structure: format, field mapping to the source data model, and reporting cadence.
  • Vendor identifies the domain expertise ensuring alignment with current federal guidance, whether through prior vendor experience, a named subject-matter collaborator, or both. Teaming with a named expert is a legitimate route, not a weakness.
  • Reporting templates and field mappings are configurable as federal reporting requirements evolve, without requiring a platform redeployment.

Requirement 6: Attribution and Instrument-Author IP

The instrument is the instrument author’s intellectual property. The data ownership framework is set between the instrument author and the funding or grantee organization through a license agreement. The vendor stewards data per that license. The vendor does not hold ownership rights and should not be in a position to assert any.9

The RFP should require:

  • The instrument license between the author and the funding or grantee organization is incorporated by reference into the contract. The vendor receives use rights through the license, not from the contract directly.
  • Attribution on every public-facing page: sponsor, funder, and the instrument author.
  • Instrument-author credit on the assessment surface itself and in the header of any data export.
  • Attribution placement and visual treatment approved by the funder and instrument author prior to launch.
  • Sponsor and funder attribution is configurable per deployment so that a single platform serving multiple grantees displays the correct sponsor and funder for each instance. Instrument-author credit remains fixed across all deployments and cannot be removed by deploying organizations.
  • Vendor describes the technical controls that enforce the ownership boundaries set by the license. For example, separation of identifiable data from aggregate data, scoped role-based access, and data-portability procedures aligned with license terms.

Requirement 7: End-of-Term Disposition

Sustainability is a contract negotiation, not an RFP requirement. What the RFP must require is a clear plan for what happens to the platform and the data at the end of the contracted period. Without it, decommissioning becomes ad hoc and the data is at risk.10

The RFP should require a credible end-of-term disposition plan structured as one of two paths:

  • Transfer. Platform operations, data, source artifacts, and documentation transfer to a successor steward (the funder, a research institution, or another vendor). Vendor describes the transfer procedure, timeline, and any vendor-side support during transition.
  • Decommissioning. Notification timeline to participating organizations, complete data export to the funder in documented formats, removal of sponsor and funder branding, deletion of remaining vendor-side copies, and the archival artifacts handed to the funder (codebase snapshot, deployment documentation, data dictionary, methodology, instrument-author license confirmation).

Specific sustainability terms (continuation funding model, maintenance term length, ongoing-cost structure, hosting transfer) are negotiated as part of the final contract. They depend on the actual budget at that point and should not be pre-specified at the RFP stage unless the funder is committing to keep the platform live for a defined period.

Requirement 8: Disclosure of Prior Relationships

Validated-instrument platforms often attract bidders with prior consulting or research relationships with the instrument author. This is not a problem; it is often a sign of relevant domain depth. It does become a problem if it is undisclosed.11

The RFP should require:

  • Vendors with any prior consulting, research, or advisory relationship with the instrument author or the funding organization disclose the nature and scope of that relationship in the proposal cover letter.
  • Disclosure does not disqualify a vendor. Failure to disclose may.
  • The funder or its review committee can then evaluate the relationship in context rather than discovering it after award.

This protects the funder’s procurement process and gives qualified vendors with prior relationships a clean path to bid without ambiguity.

What a Serious Vendor Response Looks Like

The requirements above describe what the RFP should ask for. The signals below describe what a response from a serious vendor looks like. The funder’s review committee can use these as a sanity check.

TopicGeneric vendor responseSerious vendor response
Citability”We provide data exports.”Names the DOI registration service. Describes snapshot cadence and immutability. Provides an example data dictionary. Names the citation format generated.
Instrument fidelity”Configurable scoring with admin permissions.”Describes architectural separation of instrument and analysis layers. Commits to versioned instrument deployment with per-record version tagging. Confirms no platform-side modification path for items, scoring, or domains.
Federal reporting”Grant reporting available.”Cites the specific frameworks by name. Describes field mapping. Identifies domain expertise: prior work, named SME collaborator, or both.
Sustainability”We will maintain and seek funding.”Commits to a defined initial maintenance term. Provides a clear end-of-term path (transfer or decommissioning). Sketches non-binding sustainability options for contract negotiation.
Data ownership”Vendor owns and licenses to the funder.”Defers to the instrument license. Describes how the platform enforces the boundaries set by the license. Does not assert ownership.
Prior relationshipSilent or vague.Discloses any prior consulting or research relationship with the instrument author or funder, with nature, scope, and dates.

A note on price. A bid that is meaningfully cheaper than the field is not a savings, it is a signal. For a validated-instrument platform, a low bid usually means the vendor has not understood requirements 1, 2, 5, or 6. The funder pays the actual cost later, in lost citability, eroded instrument fidelity, or platform decommissioning under duress. The RFP should weight cost modestly relative to technical approach, demonstrated experience, and prototype demonstration.

What This Page Does Not Cover

This page is about what the RFP should require. It does not cover:

  • The internal architecture choices a vendor makes to meet these requirements. Multiple defensible architectures exist; vendors should describe their own.
  • Specific cost structures or pricing benchmarks. These belong in the cost proposal section of each vendor response and are evaluated relative to scope.
  • Procurement-code specifics (insurance, bonds, certifications) that vary by jurisdiction. Funder procurement staff should add these per local requirements.
  • Instrument-specific scoring, item content, or normative data. These come from the instrument author and are attached to the contract by reference rather than reproduced in the RFP.

Closing

An RFP that requires the eight items above will not guarantee a perfect outcome, but it will substantially raise the floor. It will filter out vendors who are bidding a survey form with a logo on it. It will give serious vendors something concrete to respond to. And it will give the instrument author and the funder a defensible record of having asked for what the field actually needs.

The goal of the RFP is not to choose a vendor. It is to protect the instrument, the data, and the community of researchers who will depend on both for the next decade.

A one-page checklist version of this document is available at /articles/citable-research-platforms-checklist/.

Sources

  1. American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (2014). Standards for Educational and Psychological Testing. Washington, DC: American Educational Research Association. The Standards establish that validity is a property of the interpretations and uses made of test scores, not of the test itself, and that any modification to items, scoring, administration, or interpretation requires a corresponding revalidation argument. https://www.testingstandards.net/. APA overview: https://www.apa.org/science/programs/testing/standards
  2. FAIR Principles. GO FAIR Initiative. https://www.go-fair.org/fair-principles/. The principles were originally published as Wilkinson, M. D. et al. (2016). The FAIR Guiding Principles for scientific data management and stewardship. Scientific Data, 3, 160018. https://doi.org/10.1038/sdata.2016.18
  3. Zenodo, operated by CERN and OpenAIRE, issues DataCite-registered DOIs free of charge for research datasets. https://zenodo.org/. DataCite is a global DOI registration agency for research outputs. https://datacite.org/
  4. DDI-Codebook (DDI 2.5), DDI Alliance. An XML-based standard for documenting the content, meaning, provenance, and access of a single dataset, with strong applicability to survey and instrument metadata. https://ddialliance.org/Specification/DDI-Codebook/
  5. Sweeney, L. (2002). k-anonymity: a model for protecting privacy. International Journal on Uncertainty, Fuzziness and Knowledge-based Systems, 10(5), 557–570. Data Privacy Lab. https://dataprivacylab.org/dataprivacy/projects/kanonymity/
  6. Web Content Accessibility Guidelines (WCAG) 2.2. W3C Recommendation, December 12, 2024. https://www.w3.org/TR/WCAG22/
  7. U.S. Department of Health and Human Services. Covered Entities and Business Associates. https://www.hhs.gov/hipaa/for-professionals/covered-entities/index.html. Definitions at 45 CFR 160.103, eCFR. https://www.ecfr.gov/current/title-45/subtitle-A/subchapter-C/part-160/subpart-A/section-160.103
  8. SAMHSA grant programs referenced: Community Mental Health Services Block Grant (MHBG). https://www.samhsa.gov/grants/block-grants/mhbg. ReCAST: Resiliency in Communities After Stress and Trauma. https://www.samhsa.gov/mental-health/trauma-violence/recast. SPARS: SAMHSA Performance Accountability and Reporting System. https://spars.samhsa.gov/
  9. National Institutes of Health. NIH Data Management and Sharing Policy (effective January 25, 2023). The policy formalizes expectations for data sharing, repository selection, and responsible stewardship of data generated by NIH-funded research, and establishes the regulatory baseline against which instrument-license and data-ownership terms in federally-funded projects are negotiated. https://grants.nih.gov/policy-and-compliance/policy-topics/sharing-policies/dms
  10. Office of Management and Budget. 2 CFR 200.334, Record retention requirements. Federal grant recipients must retain records for three years from the date of submission of the final financial report, with extensions for ongoing audits, litigation, or program-income reporting. This sets the regulatory floor for end-of-term data retention and disposition planning in federally-funded projects. eCFR: https://www.ecfr.gov/current/title-2/subtitle-A/chapter-II/part-200/subpart-D/subject-group-ECFR4acc10e7e3b676f/section-200.334
  11. Office of Management and Budget. 2 CFR 200.318, General procurement standards. Federal grant recipients must maintain written standards of conduct covering conflicts of interest in contract selection, award, and administration, including organizational conflicts arising from prior relationships. https://www.ecfr.gov/current/title-2/subtitle-A/chapter-II/part-200/subpart-D/subject-group-ECFR45ddd4419ad436d/section-200.318. For investigator-side financial conflict of interest in research projects, see also National Institutes of Health Financial Conflict of Interest Policy under 42 CFR Part 50 Subpart F. https://grants.nih.gov/grants/policy/coi/