GSoC 2025

Introduction⌗

This summer I’ve had the incredible opportunity of contributing to libssh, an open-source C library for the SSHv2 protocol, as part of Google Summer of Code 2025. My project focuses on adding client-side support for FIDO/U2F keys, working under the mentorship of Eshan Kelkar, Jakub Jelen, and Sahana Prasad.

In this blog post, I’ll try to explain in detail what my project is about, the progress that has been made so far, and what lies ahead in this journey. So, let’s begin by discussing some background information required to better understand my project.

Background⌗

How Traditional SSH Keys Work⌗

To understand why FIDO keys are special, we need to first understand how SSH public key authentication works.

The following is the authentication flow with a regular public key:

You generate a key pair on your computer (like ssh-keygen -t ecdsa)
The private key stays on your computer (usually in ~/.ssh/id_ecdsa)
You copy the public key to servers you want to access (stored in ~/.ssh/authorized_keys)
When you connect to the server, it requires the client to send a valid signature
Your computer uses the private key file (i.e., ~/.ssh/id_ecdsa) to generate a valid signature and sends it to the server
The server verifies the signature using the stored public key

The problem with this is that the private key is just a file on your computer. If someone gains access to your computer or steals this file, they can impersonate you on any server that trusts your public key. Even if the private key is password-protected, a determined attacker with enough time and resources might crack it.

Enter FIDO/U2F Keys: Hardware-Backed Security⌗

FIDO (Fast Identity Online) and U2F (Universal 2nd Factor) keys are small devices, like USBs, that revolutionize this security model. Instead of storing your private key as a file on your computer, the FIDO device uses a master key stored in its dedicated security chip to deterministically derive private keys.

The following are a few reasons why FIDO keys are fundamentally more secure:

Hardware Protection: The master key is stored inside a tamper-resistant hardware security module. It literally cannot be extracted from the device! The private key derivation process also happens entirely within the security chip, making it impossible for malware or hackers to obtain any key material remotely.
Physical Presence Required: Most FIDO keys require you to physically touch or interact with the device to authorize each authentication (see this section). This means even if someone compromises your computer, they can’t authenticate without physically accessing your FIDO key.
No Shared Secrets: Unlike passwords or even traditional private keys, FIDO keys use a challenge-response protocol where no secret information like a password is ever transmitted over the network. Each authentication is unique and cannot be replayed by an attacker.

This makes them much safer than passwords, SMS codes, or even traditional SSH keys, as they require an attacker to have both remote access to your computer AND physical access to your FIDO device - a significantly higher bar for compromise.

I highly recommend you check out this cool demonstration of the authentication flow using a FIDO device.

Now that we understand the basics of FIDO keys, let’s discuss some of the key concepts related to FIDO keys that are relevant to my project.

Resident Keys⌗

FIDO devices can store credentials in two fundamentally different ways: non-resident keys and resident keys (also called discoverable credentials). In the context of SSH authentication:

Non-Resident Keys: With regular FIDO credentials, a credential ID (also known as a key handle) and associated metadata are stored in a file on the client side. When authentication is needed, the client provides the credential ID to the FIDO device, which then derives the corresponding private key from its master key to authenticate.

Resident Keys (Discoverable Credentials): With resident keys, the private key and associated metadata are stored directly in persistent memory on the FIDO device itself and nothing is stored on the client side. During authentication, the device can present all available credentials without needing to know any credential ID.

The following are some advantages of using resident keys:

Portability: The same FIDO device can be used with multiple client devices, without needing to copy any files.
Disaster Recovery: If your computer crashes or is lost, you can immediately regain access to all your SSH servers using just your FIDO device on any other computer.

You can read more about resident keys here.

User Presence and User Verification⌗

FIDO authentication incorporates two important security concepts that provide different levels of assurance and user experience. A relying party (like an SSH server) can require either or both of these features during authentication to ensure additional security:

User Presence (UP): Confirms that a human is physically interacting with the FIDO device, typically by touching a button or sensor. This doesn’t identify who the user is, but ensures someone is physically controlling the device, preventing remote attacks and automated abuse. This acts as a second factor alongside traditional SSH authentication, preventing remote compromise.

User Verification (UV): Actually verifies the user’s identity through biometric authentication (fingerprint, face recognition) or a PIN entered on the device. This provides stronger security by ensuring that even if someone steals your FIDO device, they can’t use it without your biometric or PIN. This enables passwordless SSH authentication where the FIDO key serves as the sole authentication factor.

You can read more about UP and UV here.

Now that we have a good understanding of what FIDO keys are, how they work, and their key security features, let’s explore how my GSoC project brings this technology to libssh.

About the Project⌗

While libssh already supports server-side authentication using security keys, it currently does not have support for interacting with FIDO devices on the client side. This means that users can’t use their FIDO devices to authenticate to SSH servers when using libssh-based clients, which forces them to compromise on security.

My GSoC project is all about solving this problem by implementing client-side FIDO/U2F support in libssh. This involves:

1. Device Communication: Building the infrastructure for libssh to interact with FIDO devices through various communication protocols (primarily USB HID, but designed to support future protocols like NFC or Bluetooth).

2. Key Management Operations: Implementing the ability for libssh to:

Generate new FIDO keys: Create new cryptographic credentials directly on your FIDO device
Load resident keys: Discover and retrieve keys that are stored on the device itself
Import/Export keys: Support storing and loading key files containing credential IDs and metadata, similar to how traditional SSH keys work

3. Authentication Operations: Adding support for:

Digital signing: Use your FIDO device to create cryptographic signatures that prove your identity
UP/UV verification: Ensuring that authentication requires appropriate physical interaction with the device

4. Developer-Friendly API: Creating clean, easy-to-use functions that application developers can integrate into their SSH clients without needing to understand the complex underlying FIDO protocols.

Now that we’ve outlined what the project aims to achieve, let’s understand the technical approach and architecture that makes this possible.

The Plan⌗

Callback-based Architecture⌗

FIDO/U2F keys support a wide range of communication protocols including USB HID, NFC, and potentially other protocols in the future. To effectively handle this diversity and ensure extensibility, after discussion with my mentors, the following approach has been adopted, which is somewhat inspired by OpenSSH’s approach:

We mainly need to interact with FIDO devices to perform the following core operations:

Enrolling new keys
Signing some data using an existing key
Loading all the resident keys

So, to allow libssh to support different FIDO device communication methods without tying the library to any specific protocol implementation, we define a Security Key API (sk_api.h) that contains the various parameters that these operations can take, and the structs that they must return. We then model these operations as function callbacks — essentially pluggable functions that can be swapped out by the user depending on the communication protocol or hardware that is needed. This callback-based approach has already been used in other areas in libssh, so its users shouldn’t have much trouble in dealing with the newly defined callbacks:

sk_api_version_callback: For checking the version of the Security Key API being used to ensure compatibility
sk_enroll_callback: For generating new FIDO credentials on devices
sk_sign_callback: For creating cryptographic signatures during authentication
sk_load_resident_keys_callback: For discovering and loading resident keys from devices

Currently, defaults for these callbacks are provided by libssh which interact with FIDO devices over the USB HID protocol, which is by far the most common one, via the libfido2 library. Also, the parameters and return types of these callbacks are consistent with OpenSSH’s, so that any middleware developed for OpenSSH which implements the above core operations can also be used with libssh.

However, the callback system is just one part of the puzzle. We also need to integrate FIDO support into libssh’s existing key management infrastructure.

PKI Integration⌗

libssh currently has many functions as part of its internal PKI (Public Key Infrastructure) which support various operations for dealing with traditional RSA, ECDSA, Ed25519, etc., keys. To deal with the new security key types, we need to either extend the existing PKI functions, or create entirely new functions (pki_sk.c) for operations which require considerably different handling from traditional keys.

With the backend infrastructure in place, the final piece is ensuring that application developers have intuitive APIs to work with.

Client API Integration⌗

To complete support for FIDO keys on the client side, the public-facing APIs also need to be updated. After discussion with my mentors, I decided to go ahead with the following approach:

Authentication API Enhancement: The existing ssh_userauth_publickey function responsible for authenticating against an SSH server only supports traditional key types. It must be extended to seamlessly handle security key authentication. When a security key is detected, the function should automatically:

Invoke the appropriate security key callbacks for signing operations
Handle user presence and verification requirements
Manage the additional metadata that security keys require (like application parameters and credential IDs)

Key Generation API Redesign: The current ssh_pki_generate function is designed for traditional key generation and doesn’t accommodate the unique requirements of security keys, such as:

Requiring physical device interaction for key enrollment
Supporting resident vs. non-resident key creation options
Handling user presence and verification requirements during key generation

Therefore, a new, more flexible key generation API is needed that can handle both traditional and security key generation while providing fine-grained control over key parameters.

Resident Key Management APIs: Entirely new API(s) need to be created for loading resident keys, as this is a concept that doesn’t exist in traditional SSH keys.

Overall Architecture⌗

The overall architecture of the project is as follows:

┌─────────────────────────────────────┐
│          libssh Client API          │ ← User-facing SSH client functions
├─────────────────────────────────────┤
│         PKI SK Integration          │ ← pki_sk.c - adds SK capabilities to libssh's existing PKI
├─────────────────────────────────────┤
│       Security Key API Layer        │ ← sk_api.h - abstract callback interface
├─────────────────────────────────────┤
│       Protocol Implementations      │ ← sk_usbhid.c - USB HID via libfido2
├─────────────────────────────────────┤
│       Hardware Communication        │ ← libfido2, future NFC/BLE libraries
└─────────────────────────────────────┘

This architecture ensures that adding support for new protocols (like NFC or Bluetooth) only requires implementing the SK API callbacks for that specific protocol, without modifying the core libssh codebase.

The Progress So Far⌗

Over the past few weeks, I’ve made significant progress as per the above described plan. The following is a detailed breakdown of what I have worked on so far:

Build System and Dependency Management⌗

I started by adding proper build system support for the new FIDO functionality:

Created a Findlibfido2.cmake to automatically detect and link against the libfido2 library
Added a WITH_FIDO CMake build option that allows users to conditionally enable FIDO support during compilation
Updated build scripts to properly handle libfido2 dependencies and include paths
Modified the GitLab CI pipeline to enable FIDO support during automated testing

This approach ensures that the FIDO functionality integrates seamlessly with libssh’s existing build infrastructure while remaining optional for users who don’t need it.

Security Key API Framework⌗

I added the pluggable Security Key API (sk_api.h) that serves as the abstraction layer:

Defined standardized callbacks for all core operations as described earlier
Created session-level integration allowing SSH clients to register and manage SK callbacks through some newly added APIs

USB HID Backend Implementation⌗

I developed a complete USB HID backend (sk_usbhid.c) using libfido2, based on OpenSSH’s implementation, which supports the three core operations discussed above and serves as the default implementation for the defined callbacks.

With the low-level communication layer working, I then focused on integrating FIDO keys into libssh’s existing key management system.

PKI System Integration⌗

I enhanced libssh’s Public Key Infrastructure to handle security keys:

New Key Types: Added support for SSH_KEYTYPE_SK_ECDSA and SSH_KEYTYPE_SK_ED25519 key types
Key Management: Extended existing functions for key serialization, deserialization, and comparison to handle SK-specific metadata
Import/Export: Updated key import and export functionality to properly handle security key credentials and associated metadata
PKI Bridge: Created pki_sk.c as a bridge layer that connects the abstract SK API with libssh’s concrete PKI operations

The next step was ensuring that application developers have seamless access to these new capabilities.

Client-Facing API Updates⌗

Authentication API Enhancement: The existing function to authenticate using the public key method ssh_userauth_publickey has been extended to allow authenticating using security keys as well. Although as of now this API does not support passing in the PIN for the FIDO device, plans have been made to add this functionality in the coming weeks.

New Key Generation API: Recognizing the limitations of the existing ssh_pki_generate function for security keys, a new, more flexible key generation API has been added that:

Supports Security Key Enrollment: Allows applications to generate new FIDO credentials directly on connected devices
Configurable Parameters: Provides fine-grained control over key generation options including:
- User Presence and User Verification requirements
- Resident vs. Non-resident key creation
Future-Proof Design: Built with extensibility in mind to accommodate future key types

Of course, no implementation is complete without thorough testing to ensure reliability and compatibility.

Comprehensive Testing⌗

Test Infrastructure Improvement:

To allow for testing the FIDO functionality in the CI environment, where a physical FIDO device is not available, I have made use of the openssh-sk-dummy library, which provides callbacks for a dummy FIDO device that can be used for testing purposes.

Unit Tests:

torture_sk_usbhid.c has been created which thoroughly tests the internal USB HID callbacks. This can be used to test the USB HID backend and requires a physical FIDO device to be present.
torture_pki_sk.c has been written with comprehensive tests for PKI SK integration, covering key generation, serialization, signing, and validation. This test suite does not require a physical FIDO device and can be run in the CI environment.

Integration Tests:

The existing torture_auth.c test suite has been extended to include tests for the new security key authentication flow.

All tests have been integrated into the GitLab CI pipeline (except torture_sk_usbhid.c, as it requires a physical FIDO device to be present) to ensure that any future changes do not break the FIDO functionality.

You can explore all of these changes and the complete implementation details in my GitLab MR.

What Lies Ahead⌗

With the core infrastructure now in place, the remaining weeks of GSoC 2025 focus on refining the implementation and preparing it for production use. Here’s what I’m working on next:

Sign API Improvements: Currently working on enhancing the signing API to provide more flexibility and security options:

PIN Support: Adding support for passing user PINs to the authenticator for enhanced security operations
Enhanced Options Parsing: Improving the parsing and validation of signing options to support advanced FIDO features
Better Error Reporting: Refining error codes and messages to provide clearer feedback for debugging and user experience

Resident Keys API Implementation⌗

Load Resident Keys API: Writing the resident key discovery and loading functionality:

Options Parsing: Adding support for the user to pass advanced FIDO options to this API
Device Enumeration: Implementing the ability to discover all available resident keys on a device

WebAuthn Support⌗

The WebAuthn standard defines some additional fields as part of a signature, which provide additional security and functionality. I will be working on integrating WebAuthn support into the existing FIDO implementation, so that libssh can also support WebAuthn signature-based authentication.

Beyond the core functionality, I also want to make this implementation accessible and well-documented for the users of libssh.

Updating Keygen and Examples⌗

Example Applications: Creating new examples that demonstrate the new FIDO functionality
Updated SSH Client Examples: Modifying existing libssh example applications to showcase FIDO authentication
Updating libssh Keygen: Adding support for generating FIDO keys using the new key generation API in libssh’s provided keygen2.c

Documentation⌗

API Documentation: Complete documentation of all new functions and their usage using Doxygen comments
FIDO Authentication Tutorial: Creating a dedicated documentation chapter similar to libssh’s existing authentication guide, providing detailed explanations of FIDO key authentication methods, code examples, and best practices for secure implementation

Conclusion⌗

I’m incredibly grateful to my mentors for their constant guidance in shaping the direction of my project. Their advice has been invaluable. They have been very patient in reviewing my work, catching any mistakes, and providing suggestions to improve it. My journey so far has been an amazing learning experience, and I’m looking forward to finishing my project and seeing it become a part of libssh.

Thanks for reading my blog!