AgentBin — Server Compliance (TCK)

❌ Failing Tests (54)

mandatory 40 failing

Test	Category	Transport	Error
`test_push_notification_not_supported_error_32003_enhanced`	mandatory	jsonrpcrest	TypeError: transport_create_task_push_notification_config() takes from 2 to 3 positional arguments but 4 were given
`test_extended_agent_card_method_exists`	mandatory	jsonrpcrest	AssertionError: assert ('401' in '[REST] Unexpected error in REST get_extended_agent_card: [REST] REST transport error: HTTP 404: 404 page not found\n
`test_core_method_mapping_compliance`	mandatory	jsonrpcrest	Failed: Core method mapping test failed on jsonrpc: 'task'
`test_message_send_valid_text`	mandatory	jsonrpcrest	AssertionError: Message send failed: {'error': {'code': -32001, 'message': 'Task not found'}} assert False + where False = is_json_rpc_success_respo
`test_message_send_invalid_params`	mandatory	jsonrpcrest	AssertionError: Expected InvalidParamsError (-32602), got -32001 assert -32001 == -32602
`test_message_send_continue_task`	mandatory	jsonrpcrest	AssertionError: Failed to extract task ID from response assert None is not None
`test_task_history_length`	mandatory	jsonrpcrest	KeyError: 'task'
`test_list_all_tasks`	mandatory	jsonrpcrest	KeyError: 'task'
`test_list_tasks_empty_when_none_exist`	mandatory	jsonrpcrest	AssertionError: tasks/list failed: {'jsonrpc': '2.0', 'id': '2e58539c-9654-4c02-a5d1-b161065fde0d', 'error': {'code': -31401, 'message': 'unauthentica
`test_list_tasks_validates_required_fields`	mandatory	jsonrpcrest	KeyError: 'task'
`test_list_tasks_sorted_by_timestamp_descending`	mandatory	jsonrpcrest	KeyError: 'task'
`test_filter_by_context_id`	mandatory	jsonrpcrest	KeyError: 'task'
`test_filter_by_status`	mandatory	jsonrpcrest	KeyError: 'task'
`test_filter_by_last_updated_after`	mandatory	jsonrpcrest	KeyError: 'task'
`test_combined_filters`	mandatory	jsonrpcrest	KeyError: 'task'
`test_default_page_size`	mandatory	jsonrpcrest	AssertionError: tasks/list failed: {'jsonrpc': '2.0', 'id': '41adb3ee-3a4f-4088-95ec-096b5a36b739', 'error': {'code': -31401, 'message': 'unauthentica
`test_custom_page_size`	mandatory	jsonrpcrest	KeyError: 'task'
`test_page_token_navigation`	mandatory	jsonrpcrest	KeyError: 'task'
`test_last_page_detection`	mandatory	jsonrpcrest	KeyError: 'task'
`test_total_size_accuracy`	mandatory	jsonrpcrest	KeyError: 'task'
`test_history_length_zero`	mandatory	jsonrpcrest	KeyError: 'task'
`test_history_length_custom`	mandatory	jsonrpcrest	KeyError: 'task'
`test_history_length_exceeds_actual`	mandatory	jsonrpcrest	KeyError: 'task'
`test_artifacts_excluded_by_default`	mandatory	jsonrpcrest	KeyError: 'task'
`test_artifacts_included_when_requested`	mandatory	jsonrpcrest	KeyError: 'task'
`test_invalid_page_token_error`	mandatory	jsonrpcrest	AssertionError: Invalid pageToken should return -32602, got -31401 assert -31401 == -32602 + where -31401 = <built-in method get of dict object at 0
`test_invalid_status_error`	mandatory	jsonrpcrest	AssertionError: Invalid status should return -32602, got -31401 assert -31401 == -32602 + where -31401 = <built-in method get of dict object at 0x00
`test_negative_page_size_error`	mandatory	jsonrpcrest	AssertionError: Negative pageSize should return -32602, got -31401 assert -31401 == -32602 + where -31401 = <built-in method get of dict object at 0
`test_zero_page_size_error`	mandatory	jsonrpcrest	AssertionError: pageSize=0 should return -32602, got -31401 assert -31401 == -32602 + where -31401 = <built-in method get of dict object at 0x000002
`test_out_of_range_page_size_error`	mandatory	jsonrpcrest	AssertionError: pageSize > 100 should return -32602, got -31401 assert -31401 == -32602 + where -31401 = <built-in method get of dict object at 0x00
`test_default_page_size_is_50`	mandatory	jsonrpcrest	KeyError: 'task'
`test_negative_history_length_error`	mandatory	jsonrpcrest	AssertionError: Negative historyLength should return -32602, got -31401 assert -31401 == -32602 + where -31401 = <built-in method get of dict object
`test_invalid_timestamp_error`	mandatory	jsonrpcrest	AssertionError: Invalid timestamp should return -32602, got -32700 assert -32700 == -32602 + where -32700 = <built-in method get of dict object at 0
`test_extended_card_access_controls`	mandatory	jsonrpcrest	Failed: SECURITY VIOLATION: Extended Agent Card accessible without authentication. A2A v0.3.0 Section 9.1 requires authentication for extended cards.
`test_message_sending_equivalence`	mandatory	jsonrpcrest	AssertionError: Message sending failed on all transports: ['HTTP/JSON-RPC'] assert 0 > 0 + where 0 = len([])
`test_agent_card_access_equivalence`	mandatory	jsonrpcrest	AssertionError: Agent Card access failed on all transports assert 0 > 0 + where 0 = len([])
`test_concurrent_operation_equivalence`	mandatory	jsonrpcrest	AssertionError: No concurrent operations succeeded across any transport assert 0 > 0 + where 0 = len([])
`test_tasks_cancel_valid`	mandatory	jsonrpcrest	AssertionError: Failed to extract task ID from task creation response assert None is not None
`test_tasks_get_valid`	mandatory	jsonrpcrest	AssertionError: Failed to extract task ID from task creation response assert None is not None
`test_tasks_get_with_history_length`	mandatory	jsonrpcrest	AssertionError: Failed to extract task ID from task creation response assert None is not None

quality 11 failing

Test	Category	Transport	Error
`test_parallel_requests`	quality	jsonrpcrest	AssertionError: All parallel requests failed - poor concurrency handling assert 0 > 0
`test_rapid_sequential_requests`	quality	jsonrpcrest	AssertionError: Too many failures: 0 out of 10 succeeded assert 0 > (10 * 0.8)
`test_concurrent_operations_same_task`	quality	jsonrpcrest	KeyError: 'task'
`test_empty_arrays`	quality	jsonrpcrest	AssertionError: Should return InvalidParams error code assert -32001 == -32602
`test_null_optional_fields`	quality	jsonrpcrest	KeyError: 'task'
`test_unicode_and_special_chars`	quality	jsonrpcrest	KeyError: 'task'
`test_boundary_values`	quality	jsonrpcrest	KeyError: 'task'
`test_streaming_reconnection_simulation`	quality	jsonrpcrest	KeyError: 'task'
`test_partial_update_recovery`	quality	jsonrpcrest	KeyError: 'task'
`test_task_state_transitions`	quality	jsonrpcrest	KeyError: 'task'
`test_task_cancel_state_handling`	quality	jsonrpcrest	KeyError: 'task'

features 2 failing

Test	Category	Transport	Error
`test_empty_message_parts`	features	jsonrpcrest	AssertionError: Expected error code -32602 (Invalid params) or -32603 (Internal error) for empty parts, but got: -32001 (Both codes are valid for vali
`test_missing_required_message_fields`	features	jsonrpcrest	AssertionError: Expected InvalidParams error code -32602 for missing messageId, but got: -32001 (Spec: InvalidParamsError) assert -32001 == -32602

transport-equivalence 1 failing

Test	Category	Transport	Error
`test_same_error_handling_invalid_params`	transport-equivalence	cross-transport	AssertionError: Error code mismatch for invalid params: jsonrpc=-32602 vs rest=-32001 assert -32001 == -32602

✅ Passing Tests (59)

Test	Category	Transport
`test_fetch_agent_card_success`	features	jsonrpcrest
`test_fetch_agent_card_not_found`	features	jsonrpcrest
`test_fetch_agent_card_invalid_json`	features	jsonrpcrest
`test_get_sut_rpc_endpoint`	features	jsonrpcrest
`test_get_capability_streaming`	features	jsonrpcrest
`test_get_capability_push_notifications`	features	jsonrpcrest
`test_get_supported_modalities`	features	jsonrpcrest
`test_get_authentication_schemes`	features	jsonrpcrest
`test_invalid_file_part`	features	jsonrpcrest
`test_very_large_message`	features	jsonrpcrest
`test_file_part_without_mimetype`	features	jsonrpcrest
`test_reference_task_ids_invalid`	features	jsonrpcrest
`test_a2a_error_code_coverage_summary`	mandatory	jsonrpcrest
`test_a2a_error_code_coverage_enhanced`	mandatory	jsonrpcrest
`test_rejects_malformed_json`	mandatory	jsonrpc
`test_rejects_invalid_json_rpc_requests[invalid_request0--32600]`	mandatory	jsonrpc
`test_rejects_invalid_json_rpc_requests[invalid_request1--32600]`	mandatory	jsonrpc
`test_rejects_invalid_json_rpc_requests[invalid_request2--32601]`	mandatory	jsonrpc
`test_rejects_invalid_json_rpc_requests[invalid_request3--32600]`	mandatory	jsonrpc
`test_rejects_invalid_json_rpc_requests[invalid_request4--32602]`	mandatory	jsonrpc
`test_rejects_unknown_method`	mandatory	jsonrpc
`test_rejects_invalid_params`	mandatory	jsonrpc
`test_duplicate_request_ids`	mandatory	jsonrpc
`test_invalid_jsonrpc_version`	mandatory	jsonrpc
`test_missing_method_field`	mandatory	jsonrpc
`test_raw_invalid_json`	mandatory	jsonrpc
`test_transport_specific_method_naming`	mandatory	jsonrpcrest
`test_transport_compliance_validation`	mandatory	jsonrpcrest
`test_required_method_availability`	mandatory	jsonrpcrest
`test_transport_specific_features`	mandatory	jsonrpcrest
`test_comprehensive_a2a_v030_compliance`	mandatory	jsonrpcrest
`test_agent_card_available`	mandatory	jsonrpcrest
`test_mandatory_fields_present`	mandatory	jsonrpcrest
`test_mandatory_field_types`	mandatory	jsonrpcrest
`test_agent_card_mandatory_fields`	mandatory	jsonrpcrest
`test_agent_card_capabilities_mandatory`	mandatory	jsonrpcrest
`test_agent_card_skills_mandatory`	mandatory	jsonrpcrest
`test_agent_card_input_output_modes_mandatory`	mandatory	jsonrpcrest
`test_agent_card_basic_info_mandatory`	mandatory	jsonrpcrest
`test_tasks_cancel_nonexistent`	mandatory	jsonrpcrest
`test_tasks_get_nonexistent`	mandatory	jsonrpcrest
`test_invalid_method_error_validation`	mandatory	jsonrpcrest
`test_invalid_params_error_validation`	mandatory	jsonrpcrest
`test_nonexistent_resource_error_validation`	mandatory	jsonrpcrest
`test_error_consistency_across_methods`	mandatory	jsonrpcrest
`test_error_response_completeness`	mandatory	jsonrpcrest
`test_public_agent_card_access_control`	mandatory	jsonrpcrest
`test_sensitive_information_protection`	mandatory	jsonrpcrest
`test_auth_required_state_support`	mandatory	jsonrpcrest
`test_error_handling_equivalence`	mandatory	jsonrpcrest
`test_performance_equivalence`	mandatory	jsonrpcrest
`test_very_long_string`	quality	jsonrpcrest
`test_unexpected_json_types`	quality	jsonrpcrest
`test_extra_fields`	quality	jsonrpcrest
`test_identical_functionality_message_send`	transport-equivalence	cross-transport
`test_same_error_handling_task_not_found`	transport-equivalence	cross-transport
`test_error_response_structure_equivalence`	transport-equivalence	cross-transport
`test_method_mapping_compliance`	transport-equivalence	cross-transport
`test_identical_functionality_message_stream`	transport-equivalence	cross-transport

⏭️ Skipped (48)

Test	Category	Transport	Reason
`test_reference_task_ids_valid`	features	jsonrpcrest	('D:\\github\\a2aproject\\a2a-tck\\tests\\optional\\features\\test_reference_task_ids.py', 40, 'Skipped: Failed to create reference task')
`test_security_scheme_structure_compliance`	mandatory	jsonrpcrest	('D:\\github\\a2aproject\\a2a-tck\\tests\\mandatory\\authentication\\test_auth_compliance_v030.py', 50, 'Skipped: No security schemes declared')
`test_authentication_transport_consistency`	mandatory	jsonrpcrest	('D:\\github\\a2aproject\\a2a-tck\\tests\\mandatory\\authentication\\test_auth_compliance_v030.py', 145, 'Skipped: No security schemes declared')
`test_security_error_response_compliance`	mandatory	jsonrpcrest	('D:\\github\\a2aproject\\a2a-tck\\tests\\mandatory\\authentication\\test_auth_compliance_v030.py', 226, 'Skipped: No security schemes declared')
`test_oauth2_metadata_url_validation`	mandatory	jsonrpcrest	('D:\\github\\a2aproject\\a2a-tck\\tests\\mandatory\\authentication\\test_auth_compliance_v030.py', 357, 'Skipped: No security schemes declared')
`test_mutual_tls_scheme_declaration`	mandatory	jsonrpcrest	('D:\\github\\a2aproject\\a2a-tck\\tests\\mandatory\\authentication\\test_auth_compliance_v030.py', 430, 'Skipped: No security schemes declared')
`test_authentication_required_when_declared`	mandatory	jsonrpcrest	('D:\\github\\a2aproject\\a2a-tck\\tests\\mandatory\\authentication\\test_auth_enforcement.py', 56, 'Skipped: No authentication schemes declared in Ag
`test_invalid_credentials_rejected`	mandatory	jsonrpcrest	('D:\\github\\a2aproject\\a2a-tck\\tests\\mandatory\\authentication\\test_auth_enforcement.py', 150, 'Skipped: No authentication schemes declared in A
`test_authentication_scheme_consistency`	mandatory	jsonrpcrest	('D:\\github\\a2aproject\\a2a-tck\\tests\\mandatory\\authentication\\test_auth_enforcement.py', 260, 'Skipped: No authentication schemes declared in A
`test_push_notification_not_supported_error_32003`	mandatory	jsonrpc	('D:\\github\\a2aproject\\a2a-tck\\tests\\mandatory\\jsonrpc\\test_a2a_error_codes.py', 144, 'Skipped: SUT appears to support push notifications despi
`test_unsupported_operation_error_32004`	mandatory	jsonrpcrest	('D:\\github\\a2aproject\\a2a-tck\\tests\\mandatory\\jsonrpc\\test_a2a_error_codes.py', 274, 'Skipped: Could not trigger UnsupportedOperationError (-3
`test_content_type_not_supported_error_32005`	mandatory	jsonrpcrest	('D:\\github\\a2aproject\\a2a-tck\\tests\\mandatory\\jsonrpc\\test_a2a_error_codes.py', 373, 'Skipped: Could not trigger ContentTypeNotSupportedError
`test_invalid_agent_response_error_32006`	mandatory	jsonrpcrest	('D:\\github\\a2aproject\\a2a-tck\\tests\\mandatory\\jsonrpc\\test_a2a_error_codes.py', 423, 'Skipped: InvalidAgentResponseError (-32006) is an intern
`test_unsupported_operation_error_32004_enhanced`	mandatory	jsonrpcrest	('D:\\github\\a2aproject\\a2a-tck\\tests\\mandatory\\jsonrpc\\test_a2a_error_codes_enhanced.py', 241, 'Skipped: Could not trigger -32004 error - SUT h
`test_content_type_not_supported_error_32005_enhanced`	mandatory	jsonrpcrest	('D:\\github\\a2aproject\\a2a-tck\\tests\\mandatory\\jsonrpc\\test_a2a_error_codes_enhanced.py', 332, 'Skipped: Could not trigger -32005 error - SUT a
`test_invalid_agent_response_error_32006_enhanced`	mandatory	jsonrpcrest	('D:\\github\\a2aproject\\a2a-tck\\tests\\mandatory\\jsonrpc\\test_a2a_error_codes_enhanced.py', 402, 'Skipped: Could not trigger -32006 error - SUT r
`test_multi_transport_method_mapping`	mandatory	jsonrpcrest	('D:\\github\\a2aproject\\a2a-tck\\tests\\mandatory\\protocol\\test_a2a_v030_transport_compliance.py', 140, 'Skipped: Multi-transport method mapping r
`test_authentication_scheme_validation`	mandatory	jsonrpcrest	('D:\\github\\a2aproject\\a2a-tck\\tests\\mandatory\\security\\test_agent_card_security.py', 196, 'Skipped: No security schemes declared - cannot test
`test_security_scheme_consistency`	mandatory	jsonrpcrest	('D:\\github\\a2aproject\\a2a-tck\\tests\\mandatory\\security\\test_agent_card_security.py', 403, 'Skipped: No security schemes declared - cannot test
`test_certificate_chain_trust_validation`	mandatory	jsonrpcrest	('D:\\github\\a2aproject\\a2a-tck\\tests\\mandatory\\security\\test_certificate_validation.py', 88, 'Skipped: Certificate validation tests only applic
`test_hostname_verification`	mandatory	jsonrpcrest	('D:\\github\\a2aproject\\a2a-tck\\tests\\mandatory\\security\\test_certificate_validation.py', 201, 'Skipped: Certificate validation tests only appli
`test_certificate_revocation_status`	mandatory	jsonrpcrest	('D:\\github\\a2aproject\\a2a-tck\\tests\\mandatory\\security\\test_certificate_validation.py', 336, 'Skipped: Certificate revocation tests only appli
`test_invalid_certificate_rejection`	mandatory	jsonrpcrest	('D:\\github\\a2aproject\\a2a-tck\\tests\\mandatory\\security\\test_certificate_validation.py', 425, 'Skipped: Certificate rejection tests only applic
`test_certificate_security_headers`	mandatory	jsonrpcrest	('D:\\github\\a2aproject\\a2a-tck\\tests\\mandatory\\security\\test_certificate_validation.py', 532, 'Skipped: Certificate security headers only appli
`test_in_task_authentication_workflow`	mandatory	jsonrpcrest	('D:\\github\\a2aproject\\a2a-tck\\tests\\mandatory\\security\\test_in_task_authentication.py', 247, 'Skipped: Agent did not require in-task authentic
`test_authentication_challenge_headers`	mandatory	jsonrpcrest	('D:\\github\\a2aproject\\a2a-tck\\tests\\mandatory\\security\\test_in_task_authentication.py', 275, 'Skipped: No authentication schemes declared - ca
`test_invalid_authentication_handling`	mandatory	jsonrpcrest	('D:\\github\\a2aproject\\a2a-tck\\tests\\mandatory\\security\\test_in_task_authentication.py', 372, 'Skipped: No authentication schemes declared - ca
`test_auth_state_transitions`	mandatory	jsonrpcrest	('D:\\github\\a2aproject\\a2a-tck\\tests\\mandatory\\security\\test_in_task_authentication.py', 484, 'Skipped: Could not create test task for authenti
`test_tls_protocol_version_security`	mandatory	jsonrpcrest	('D:\\github\\a2aproject\\a2a-tck\\tests\\mandatory\\security\\test_tls_configuration_enhanced.py', 146, 'Skipped: TLS configuration tests only applic
`test_cipher_suite_security_analysis`	mandatory	jsonrpcrest	('D:\\github\\a2aproject\\a2a-tck\\tests\\mandatory\\security\\test_tls_configuration_enhanced.py', 228, 'Skipped: TLS configuration tests only applic
`test_certificate_chain_validation`	mandatory	jsonrpcrest	('D:\\github\\a2aproject\\a2a-tck\\tests\\mandatory\\security\\test_tls_configuration_enhanced.py', 321, 'Skipped: TLS configuration tests only applic
`test_tls_security_features`	mandatory	jsonrpcrest	('D:\\github\\a2aproject\\a2a-tck\\tests\\mandatory\\security\\test_tls_configuration_enhanced.py', 483, 'Skipped: TLS configuration tests only applic
`test_tls_implementation_best_practices`	mandatory	jsonrpcrest	('D:\\github\\a2aproject\\a2a-tck\\tests\\mandatory\\security\\test_tls_configuration_enhanced.py', 554, 'Skipped: TLS configuration tests only applic
`test_https_required_production`	mandatory	jsonrpcrest	('D:\\github\\a2aproject\\a2a-tck\\tests\\mandatory\\security\\test_transport_security.py', 81, 'Skipped: HTTPS requirement skipped for localhost/test
`test_tls_certificate_validation`	mandatory	jsonrpcrest	('D:\\github\\a2aproject\\a2a-tck\\tests\\mandatory\\security\\test_transport_security.py', 148, 'Skipped: Certificate validation relaxed for testing
`test_tls_configuration_security`	mandatory	jsonrpcrest	('D:\\github\\a2aproject\\a2a-tck\\tests\\mandatory\\security\\test_transport_security.py', 247, 'Skipped: TLS configuration check relaxed for testing
`test_http_to_https_redirect`	mandatory	jsonrpcrest	('D:\\github\\a2aproject\\a2a-tck\\tests\\mandatory\\security\\test_transport_security.py', 341, 'Skipped: HTTP redirect test skipped for localhost/te
`test_task_retrieval_equivalence`	mandatory	jsonrpcrest	('D:\\github\\a2aproject\\a2a-tck\\tests\\mandatory\\transport\\test_multi_transport_equivalence.py', 235, 'Skipped: Task creation did not return vali
`test_identical_functionality_tasks_get`	transport-equivalence	cross-transport	('D:\\github\\a2aproject\\a2a-tck\\tests\\optional\\multi_transport\\test_functional_equivalence.py', 104, 'Skipped: Cannot extract task ID from respo
`test_consistent_behavior_message_send`	transport-equivalence	cross-transport	('D:\\github\\a2aproject\\a2a-tck\\tests\\optional\\multi_transport\\test_functional_equivalence.py', 177, 'Skipped: Need at least 2 successful respon
`test_consistent_behavior_tasks_get`	transport-equivalence	cross-transport	('D:\\github\\a2aproject\\a2a-tck\\tests\\optional\\multi_transport\\test_functional_equivalence.py', 215, 'Skipped: Cannot extract task ID from respo
`test_identical_functionality_tasks_cancel`	transport-equivalence	cross-transport	('D:\\github\\a2aproject\\a2a-tck\\tests\\optional\\multi_transport\\test_functional_equivalence.py', 281, 'Skipped: Cannot extract task ID from respo
`test_consistent_behavior_tasks_cancel`	transport-equivalence	cross-transport	('D:\\github\\a2aproject\\a2a-tck\\tests\\optional\\multi_transport\\test_functional_equivalence.py', 325, 'Skipped: Cannot extract task ID from respo
`test_same_error_handling_method_not_found`	transport-equivalence	cross-transport	('D:\\github\\a2aproject\\a2a-tck\\tests\\optional\\multi_transport\\test_functional_equivalence.py', 473, 'Skipped: Method not found error testing re
`test_same_error_handling_task_not_cancelable`	transport-equivalence	cross-transport	('D:\\github\\a2aproject\\a2a-tck\\tests\\optional\\multi_transport\\test_functional_equivalence.py', 535, 'Skipped: Cannot extract task ID from respo
`test_equivalent_authentication_agent_card`	transport-equivalence	cross-transport	('D:\\github\\a2aproject\\a2a-tck\\tests\\optional\\multi_transport\\test_functional_equivalence.py', 720, 'Skipped: Need at least 2 agent card respon
`test_identical_functionality_tasks_resubscribe`	transport-equivalence	cross-transport	('D:\\github\\a2aproject\\a2a-tck\\tests\\optional\\multi_transport\\test_functional_equivalence.py', 868, 'Skipped: Cannot extract task ID from respo
`test_streaming_response_equivalence`	transport-equivalence	cross-transport	('D:\\github\\a2aproject\\a2a-tck\\tests\\optional\\multi_transport\\test_functional_equivalence.py', 965, 'Skipped: Need at least 2 streaming respons

🧪 Server Compliance — A2A TCK

❌ Failing Tests (54)

✅ Passing Tests (59)

⏭️ Skipped (48)

⚠️ Errors (3)