CipherStream Enterprise Data Extraction Platform
What made this section unhelpful for you?
On this page
- CipherStream Enterprise Data Extraction Platform
Data Security and Decryption
On this page
- Data Security and Decryption
Webhook Signature Verification
Every webhook delivery includes an HMAC-SHA256 signature for authenticity verification. Always verify signatures before processing webhook payloads.
Signature Header
Verification Steps
- Extract the signature from
X-CipherStream-Signatureheader (removesha256=prefix) - Get the raw request body (do not parse JSON first)
- Compute HMAC-SHA256 using your webhook secret
- Compare signatures using constant-time comparison
Reference Implementations
Python
import hmac
import hashlib
def verify_signature(payload: bytes, signature: str, secret: str) -> bool:
expected = hmac.new(
secret.encode('utf-8'),
payload,
hashlib.sha256
).hexdigest()
return hmac.compare_digest(f"sha256={expected}", signature)
# Usage
raw_body = request.body # Raw bytes, not parsed JSON
signature = request.headers.get('X-CipherStream-Signature')
if not verify_signature(raw_body, signature, your_webhook_secret):
return Response(status=401)Node.js
const crypto = require('crypto');
function verifySignature(payload, signature, secret) {
const expected = 'sha256=' + crypto
.createHmac('sha256', secret)
.update(payload, 'utf8')
.digest('hex');
return crypto.timingSafeEqual(
Buffer.from(expected),
Buffer.from(signature)
);
}
// Usage
const rawBody = req.rawBody; // Raw string, not parsed
const signature = req.headers['x-cipherstream-signature'];
if (!verifySignature(rawBody, signature, yourWebhookSecret)) {
return res.status(401).send('Invalid signature');
}C# (.NET)
Secret Rotation
CipherStream supports zero-downtime webhook secret rotation with dual-secret validation.
- Grace Period: When a secret is rotated, both the old and new secrets are valid for a configurable grace period (default: 7 days)
- Seamless Transition: Update your verification code to accept either secret during rotation
- Rotation Notification: You'll receive notification before rotation occurs
Dual-Secret Verification Example (Python)
def verify_with_rotation(payload: bytes, signature: str,
current_secret: str, previous_secret: str = None) -> bool:
# Try current secret first
if verify_signature(payload, signature, current_secret):
return True
# Fall back to previous secret during rotation
if previous_secret and verify_signature(payload, signature, previous_secret):
return True
return FalseSecurity Best Practices
Practice | Description |
Constant-time comparison | Always use timing-safe comparison functions |
Verify before processing | Reject requests with invalid signatures immediately |
Use raw payload | Verify against raw bytes, not parsed/reformatted JSON |
Secure secret storage | Store webhook secrets in a secrets manager |
Monitor failures | Alert on repeated signature verification failures |
Retry Behaviour
Attempt | Delay | Total Wait |
1 | Immediate | 0s |
2 | 1 minute | 1m |
3 | 5 minutes | 6m |
4 | 15 minutes | 21m |
5 | 1 hour | 1h 21m |
6 | 2 hours | 3h 21m |
Webhooks are retried on 5xx errors and timeouts. Return 2xx to acknowledge successful receipt.
What made this section unhelpful for you?
On this page
- Webhook Signature Verification
1. Patient Management
Patient Management
Core patient data and analytics for comprehensive patient relationship management.
Characteristics:
- Patient demographics - Core patient information and contact details
- Patient analytics - Lifetime value and business intelligence
- Follow-up management - Recall scheduling and tracking
Use Cases:
- Patient management systems
- Contact information updates
- Demographics analysis
- Marketing and communication
- Patient retention analysis
- Follow-up scheduling
On this page
- 1. Patient Management
2. Scheduling & Appointments
Scheduling & Appointments
Complete appointment management including scheduling, status tracking, and optimisation.
Characteristics:
- Core appointments - Main appointment records with intelligent processing
- Status management - Appointment statuses and types for workflow control
- Schedule optimisation - Cancellation tracking and waitlist management
- Communication - Reminder and notification systems
Use Cases:
- Appointment booking systems
- Schedule management
- Cancellation analysis
- Waitlist optimisation
- Patient communication
- Scheduling templates
On this page
- 2. Scheduling & Appointments
3. Financial Management
Financial Management
Comprehensive financial data including transactions, payments, invoicing, and accounts receivable.
Characteristics:
- Core financials - Main financial transaction records
- Payment processing - Receipts, deposits, and payment allocations
- Billing management - Invoices and discount tracking
- Accounts receivable - Debtor management and outstanding balances
- Pricing structure - Fee schedules and payment types
Use Cases:
- Financial reporting and analysis
- Revenue tracking
- Accounting system integration
- Business intelligence dashboards
- Payment reconciliation
- Debt collection workflows
On this page
- 3. Financial Management
4. Clinical & Treatment
Clinical & Treatment
Clinical data management including completed treatments and treatment planning.
Characteristics:
- Treatment records - Completed procedures and clinical work
- Treatment planning - Proposed treatments and care plans
- Service catalogue - Available procedures and service items
Use Cases:
- Clinical reporting and analysis
- Treatment outcome tracking
- Care plan management
- Service utilisation analysis
- Clinical decision support
- Quality assurance
On this page
- 4. Clinical & Treatment
5. Business Operations
Business Operations
Business expense management and operational cost tracking.
Characteristics:
- Expense tracking - Business expense records with date filtering
- Expense categorisation - Expense categories for financial reporting
Use Cases:
- Expense tracking and reporting
- Budget analysis
- Tax preparation
- Cost centre reporting
- Financial planning
- Operational efficiency analysis
On this page
- 5. Business Operations
6. Practice Setup & Configuration
Practice Setup & Configuration
Practice management configuration including staff, locations, and system setup.
Characteristics:
- Staff management - Practitioners and users with role definitions
- Location management - Practice locations and facilities
- System configuration - User roles and access control
- External relationships - Health funds, third parties, and referral sources
- Recall management - Recall types and follow-up configurations
Use Cases:
- Practice setup and configuration
- Staff management systems
- Location and facility management
- User access control
- External partner integration
- Follow-up system configuration
On this page
- 6. Practice Setup & Configuration
7. Advanced Tools
Advanced Tools
Advanced data extraction capabilities for power users and custom requirements.
Characteristics:
- Direct table access - Extract from any accessible database table
- CALL syntax - Advanced stored procedure execution
- Extended timeout - 180-second timeout for large extractions
- Flexible parameters - Custom table names and date filtering
Use Cases:
- Custom data extractions
- Ad-hoc reporting requirements
- Data migration projects
- Specialised analytics queries
- Direct database access
- Custom integration needs
On this page
- 7. Advanced Tools
8. Job Management
Job Management
Background job management for large data extractions and processing with comprehensive monitoring and control.
When Jobs Are Created:
- Large datasets: >100,000 rows automatically trigger job processing
- File size threshold: >50MB estimated output size
- Explicit job mode: User specifically requests job processing
- System load balancing: High system load triggers job queuing
- Complex queries: Resource-intensive extractions
- Scheduled extractions: Automated recurring data pulls
Job Lifecycle States:
- Queued - Job created and waiting for available processing resources
- Running - Data extraction in progress with real-time progress updates
- Completed - Data successfully extracted and available for download via S3
- Failed - Error occurred during processing with detailed error information
- Cancelled - Job manually cancelled by user or system timeout
Job Processing Features:
- Real-time progress tracking - Live percentage completion (0-100%)
- Row count monitoring - Current number of processed records
- Time estimation - Estimated completion time based on current progress
- Resource allocation - Dedicated processing resources for optimal performance
- Error handling - Detailed error messages and recovery suggestions
- Automatic retries - Built-in retry logic for transient failures
Download Management:
- S3 secure storage - Enterprise-grade cloud storage with encryption
- Presigned URLs - Time-limited, secure download links
- 12-hour expiry - URLs automatically expire for security
- Resume support - Partial download recovery for large files
- CDN acceleration - Global content delivery for faster downloads
- Bandwidth optimisation - Compressed files for efficient transfer
Monitoring & Notifications:
- Webhook integration - Real-time job completion alerts
- Email notifications - Optional email alerts for job status changes
- Progress callbacks - Periodic progress updates via webhooks
- Performance metrics - Execution time, throughput, and resource usage
- Audit logging - Complete job history and user actions
Security & Compliance:
- Data encryption - AES-256-GCM encryption for stored files
- Access control - Customer-specific job isolation
- Audit trail - Complete job lifecycle logging
- Automatic cleanup - Files removed after expiry for data protection
- IP restrictions - Optional IP-based access control
On this page
- 8. Job Management
9. Integrations
Integrations
Advanced webhook system for real-time notifications, system integration, and automated workflow triggers.
Supported Events:
job.completed- Job finished successfully with download URL and metadatajob.failed- Job encountered an error with detailed failure informationjob.cancelled- Job was manually cancelled or timed outjob.progress- Periodic progress updates during job execution (optional)system.maintenance- Scheduled maintenance notificationsapi.rate_limit- Rate limit threshold warnings
Webhook Security Features:
- HMAC-SHA256 signature - Cryptographic payload verification using shared secret
- Timestamp validation - Prevents replay attacks with time-based verification
- IP whitelisting - Optional source IP restrictions for enhanced security
- TLS encryption - All webhook deliveries use HTTPS/TLS 1.3
- Signature verification - Complete payload integrity checking
Delivery & Reliability:
- Automatic retries - Up to 3 retry attempts with exponential backoff
- Delivery tracking - Complete success/failure monitoring and logging
- Timeout handling - 30-second response timeout with configurable settings
- Dead letter queue - Failed deliveries stored for manual retry
- Circuit breaker - Automatic endpoint disabling for persistent failures
- Rate limiting - Configurable delivery rate limits to prevent overwhelming
Webhook Payload Structure:
{
"event": "job.completed",
"timestamp": "2024-09-26T10:02:15Z",
"signature": "sha256=abc123def456...",
"delivery_id": "del_1758630144658",
"attempt": 1,
"data": {
"job_id": "appointments_1758630144658_c7511999",
"customer_id": "your-customer-id",
"procedure_name": "appointments",
"status": "completed",
"rows_processed": 45678,
"execution_time_seconds": 64.75,
"file_size_bytes": 2048576,
"s3_url": "https://secure-download-url",
"expires_at": "2024-09-26T22:02:15Z",
"output_format": "ndjson",
"compression": "gzip",
"metadata": {
"from_date": "2024-01-01",
"to_date": "2024-12-31",
"date_modifier": "Created"
}
}
}
Signature Verification:
import hmac
import hashlib
def verify_webhook_signature(payload, signature, secret):
expected_signature = hmac.new(
secret.encode('utf-8'),
payload.encode('utf-8'),
hashlib.sha256
).hexdigest()
return hmac.compare_digest(f"sha256={expected_signature}", signature)
Configuration Options:
- Event filtering - Subscribe to specific events only
- Custom headers - Add custom HTTP headers to webhook requests
- Retry configuration - Customize retry attempts and backoff strategy
- Timeout settings - Configure response timeout values
- Batch delivery - Group multiple events into single webhook call
Monitoring & Debugging:
- Delivery logs - Complete webhook delivery history
- Response tracking - HTTP status codes and response times
- Error analysis - Detailed failure reasons and troubleshooting
- Performance metrics - Delivery success rates and latency statistics
- Test endpoints - Webhook testing and validation tools
Use Cases:
- Automated workflows - Trigger business processes when jobs complete
- Real-time notifications - Instant alerts when data extraction finishes
- System integration - Connect CipherStream to other business systems
- Data pipeline automation - Chain multiple data processing steps
- Monitoring and alerting - Track job completion and system health
- Business intelligence - Trigger report generation and dashboard updates
- Customer notifications - Inform end users when their data is ready
On this page
- 9. Integrations
10. Documentation
Documentation
API documentation and OpenAPI schemas for integration and development.
Available Documentation:
- Interactive Docs - Swagger UI for testing endpoints
- OpenAPI Schema - Machine-readable API specification
- Integration Guides - Code examples and best practices
Use Cases:
- Interactive API testing and exploration
- Code generation for client libraries
- API documentation for development teams
- Integration planning and validation
- Request/response format reference
On this page
- 10. Documentation