Privacy preserving vertical distributed learning for health data

Vertical federated learning

Vertical federated learning (VFL) is a ML technique that empowers organizations with vertically partitioned data to develop and train a decentralized ML model while safeguarding data privacy and security. In this approach, data samples are distributed among different parties, with each party possessing its unique set of features or attributes. The main objective of VFL is to create a global ML model using these vertically partitioned data samples while ensuring that the raw data remains confidential and secure. To achieve this, each party independently trains its local model based on its own data and later combines the results at a central server to construct the global model. This method proves effective in scenarios where organizations have data silos and desire to harness ML to improve models without sharing raw data with one another. In VFL, data is partitioned in a manner where two nodes may share the same user profiles but contain different feature information. These nodes could represent various health institutions or providers of healthcare data applications. The aim of VFL in such cases is to create a comprehensive model by aggregating patient features from multiple institutions without directly exchanging patient data. This way, VFL ensures collaboration and model improvement while maintaining strict data privacy and security. Each node sharing the same sample of data $$ I $$ and contributing its own unique set of patient features $$ X $$ and labels $$ Y $$ information, VFL can be denoted as the following:

In this scenario, we are considering a homogenous label setting where all parties are trying to predict the same target variable, but each party may have access to different features that are relevant to the prediction. For example, from each client, the same data sample $$ I $$ can be denoted with different features $$ \{x_1^i, x_2^i, x_3^i\} $$ and $$ \{x_3^j, x_4^j, x_5^j\} $$ but with the same label $$ y $$. Consider a scenario where a hospital and a nearby immunization center are two distinct healthcare institutions operating in the same area. As they serve local populations, the patients utilizing these facilities may exhibit significant similarities. However, the data they retain could differ, with vaccine centers storing users' immunization histories and hospitals maintaining medical treatment histories. Due to this disparity in data storage, the user features may not be directly related. In this context, VFL provides a safe and effective approach to integrating diverse feature sets to enhance model performance while still preserving the locality of the data. By utilizing VFL, valuable insights can be gleaned from both types of healthcare institutions without compromising data privacy, enabling improved healthcare decision-making for the benefit of the community.

LSTM neural networks

A NN is a network of interconnected processing "nodes" or units functioning analogously to biological neurons. These nodes, synthetic versions of biological neurons, receive inputs that are multiplied by weights and then delivered to the cell body, where they are combined through basic arithmetic to produce node activations. A threshold logic unit (TLU) carries out this calculation, resulting in an output of either zero or one. Constructing a NN involves defining its model structure, including the number of input features and outputs, and initializing its parameters before running them in a loop ^{[14, 15]}. During this process, forward propagation calculates the current loss, backward propagation computes the current gradient, and gradient descent updates the parameters. Preparing the dataset and tuning the learning rate can significantly influence the algorithm's performance.

Recurrent NNs (RNNs) are a specialized type of NN that allows information to be propagated from one step of the network to the next. They are particularly suited for tasks involving sequential data, such as language translation and speech recognition. RNNs process input sequences element by element while maintaining an internal state that encodes the context of the sequence up to that point. This enables the network to utilize information from earlier elements in the sequence when processing later ones, facilitating the capture of dependencies between elements in the input sequence. RNNs come in various forms, such as LSTM networks and gated recurrent units (GRUs).

LSTM is a specific type of RNN well-suited for modeling temporal data with long-range dependencies. RNNs process sequential data by iterating through the time steps of the input and maintaining a hidden state that carries information about the past. LSTMs, as a variant of RNNs, incorporate an additional "memory cell" capable of storing information for an extended duration. Additionally, they feature three "gate" mechanisms (input, output, and forget gates) that control access to and modification of the memory cell, enabling effective long-term information retention. The equations for the forward pass of an LSTM cell can be denoted below:

Forget gate's compact forms:

Update gate's activation vector:

Forget gate's activation vector:

Output gate's activation vector:

Finally, cell state vector $$ c^t $$:

These are the standard equations employed to update the hidden state and memory cell in an LSTM at each time step. $$ x^{ < t > } $$ represents the input at time step $$ t $$, $$ a^{ < t > } $$ denotes the hidden state at time step $$ t $$, and $$ a^{ < t-1 > } $$ represents the hidden state at the previous time step. The weight matrices, $$ W_u $$, $$ W_f $$, and $$ W_o $$, correspond to the update, forget, and output gates, respectively, while $$ b $$ represents the bias term. The function $$ \sigma(x) $$ denotes the sigmoid function, which maps a value to the range [0, 1]. Additionally, the function $$ tanh(x) $$ denotes the hyperbolic tangent function, which maps a value to the range [-1, 1]. These mathematical operations play a crucial role in the LSTM's ability to update and control its hidden state and memory cell, allowing for effective memory retention and handling of sequential data. A detailed explanation of the LSTM algorithm can be found in ^[16–18].

Differential privacy

DP is a mathematical framework for protecting the privacy of individuals in a dataset. Its goal is to enable the release of useful statistical information about a dataset while ensuring that no individual's data can be inferred from the released information. The privacy level is controlled by the epsilon variable, which can be changed in accordance with varied circumstances and security guidelines. The system provides increased security as epsilon changes.

Definition 1 ($$ \varepsilon $$-Differential Privacy)An algorithm $$ X $$ is differentially private^[12] if, for any datasets $$ D $$ and $$ D' $$ differing by at most a single record, and for all sets $$ S $$$$ \in $$$$ R $$, where $$ R $$ is the range of $$ X $$, the following condition holds

Two datasets, in this instance, are said to be neighbors if their sole difference is one record. Here, the privacy budget of the method can be represented by the non-negative parameter $$ \varepsilon $$. The privacy budget determines the trade-off between the accuracy of the query and the privacy of the individuals in the dataset. A smaller privacy budget corresponds to a larger amount of noise added and, therefore, a lower accuracy but stronger privacy guarantees.

One common algorithm for achieving DP is the Laplace mechanism. This mechanism adds noise to the output of a statistical query in a way that preserves DP. The amount of noise added is determined by the "sensitivity" of the query, which measures how much the output of the query can change when data from a single individual is removed from the dataset.

Distributed learning and privacy attacks

FL is a distributed learning method that avoids sharing sensitive raw data to preserve data privacy. Recent studies have demonstrated the advantages of using FL over traditional centralized ML models, especially for sensitive data applications. However, in this distributed framework, there are potential privacy attacks, such as inference ^[6], reconstruction ^{[19, 20]}, or backdoor ^{[21, 22]} attacks, which could lead to the retrieval of data through exposed information. To address these concerns, Rajkumar et al. ^[23] proposed the use of DP algorithms to effectively minimize privacy attacks while maintaining reasonable utility. Choudhury et al. ^[7] applied this DP approach in the FL architecture for healthcare data. However, they also found that the DP noise introduced could adversely affect data utility. In their subsequent work ^[8], Choudhury et al. aimed to strike a balance between privacy and utility by implementing tailored k-anonymity to reduce DP noise as much as possible. An empirical assessment of the health records of one million patients demonstrated robust model performance that outperformed standard DP approaches. Similarly, in our recent work ^[9], we were motivated by the idea of "data sensitization first, noise applied last" to minimize DP noise in model data and enhance the overall data utility of the framework. Our approach seeks to achieve a more optimal balance between privacy protection and maintaining meaningful utility in the context of distributed healthcare data analysis.

Privacy-preserving ML can be achieved through encryption ^[24–26] or secret sharing ^[27–29] via secure multiparty communication. In these frameworks, models are communicated using encrypted data or shared secrets. However, encryption requires more computational resources and may not be suitable for all scenarios. Additionally, secure multiparty communication necessitates frequent data transfers between clients, leading to communication overhead.

In recent years, distributed ML techniques focusing on horizontal data have received increasing attention. In contrast, our work enables feature-parallel ML among nodes with vertically partitioned data, which is equally significant but has not yet been extensively investigated. This approach addresses the challenges unique to vertically distributed data, opening up new avenues for privacy-preserving ML in a different context.

Privacy-preserving distributed learning on vertical data

Numerous research efforts have been dedicated to developing privacy-preserving techniques for VDL. The primary goal of these techniques is to safeguard the privacy of training data while ensuring effective model training. One common approach involves the application of DP ^{[12, 30]}, where noise is added to the data in a manner that preserves individual privacy while enabling the model to capture meaningful patterns. Other methods include leveraging secure multiparty computation (SMC) ^[31] to facilitate joint computation of the model by multiple parties without revealing their respective data to each other. Additionally, homomorphic encryption is utilized to encrypt the data, allowing the model to be trained on the encrypted data without exposing the underlying data itself. These privacy-preserving techniques contribute to a more secure and efficient VDL process.

Liu et al. ^[10] introduced the Federated Stochastic Block Coordinate Descent (FedBCD) algorithm, which utilizes an SMC approach. In their work, each party performs multiple local updates to minimize communication overhead. The focus is on vertically partitioned data, where only a single value is shared instead of the entire model or raw data, ensuring data privacy is maintained. Similarly, Hu et al. ^[32] explored a similar approach for VFL using the Alternating Direction Method of Multipliers (ADMM), commonly employed in distributed ML. In their method, they shared a single value for model training and incorporated the $$ \varepsilon, \delta $$- DP algorithm to perturb the shared value using Gaussian noise. The perturbed approach ensures that the probability distribution of communicated values remains largely unaffected by modifications to any individual feature in a party's local dataset. With theoretical privacy guarantees, their method converges with significantly fewer epochs compared to the state-of-the-art SGD approach. Chen et al. ^[11] also proposed a VFL approach in an asynchronous manner, utilizing SMC for communication between parties. They incorporated "perturbed local embedding" with Gaussian DP noise to enhance data privacy. Their experiments covered both LR and deep learning for healthcare data, demonstrating the efficacy of their approach in maintaining privacy while training ML models on distributed features.

In a VFL scenario, effective communication of each partner's unique feature information is crucial for joint learning and training of any classification algorithm. Currently, many technologies rely on SMC techniques to exchange feature information with clients. Some strategies employ homomorphic encryption or DP for data perturbation or sanitization to preserve privacy while sharing feature information. Hu et al. ^[13] proposed a feature-distributed collaborative learning strategy where each client undergoes independent training, and the final prediction output is shared with a central server for computing the final score. This approach maintains data locality in an asynchronous SGD method by making predictions solely based on local features in a parallel computing fashion. They also employed Laplacian DP noise to safeguard the shared prediction sent to the main server. However, their technique did not specifically target healthcare data and utilized the NN deep learning technique, with LSTM-based approaches showing better performance. Our approach is quite similar to theirs, except that we adopted the latest LSTM methods specifically designed for healthcare data. By leveraging these advanced techniques, our approach demonstrates improved performance and effectiveness in the context of healthcare data analysis within the VFL framework.

Problem overview

In the proposed architecture of VFL, depicted in Figure 1, models are trained locally based on the available features at each site. The central server acts as an aggregator for the overall predictions but does not engage in any training. The complete model is trained using mini-batched SGD ^{[2, 30, 33]} at each local party. Once local training is completed, the final results are shared with the central server. To classify the local feature set and produce a local prediction, LR and LSTM NNs are employed. The central server aggregates these local predictions to generate the final prediction. To determine the appropriate feature weight for each feature, random value assignment is used. This weight matrix, together with the local prediction results, is aggregated by the central server. This design eliminates the need for communication between parties, thereby reducing the risk of data exposure. In addition to the weight-based aggregation, the local predictions are perturbed with DP noise, creating a more robust privacy-preserving model, albeit resulting in a slight decrease in accuracy in the final outcome.

Figure 1. Multiple Data Owners are training a model collaboratively using distributed feature learning.

The model performance will be evaluated using an appropriate evaluation metric, aiming to minimize the error between the predicted outputs and the true labels. The objective is to build a model that generalizes well to unseen data and can be effectively deployed in a real-world healthcare setting to address the problem at hand.

Local prediction calculation

In our model, each data source or site contains a unique feature set. As a result, we train our model based solely on the available local features in this step. In a similar setting, Hu et al. ^[13] utilized LR and convolutional NN (CNN) approaches in mobile app datasets. In contrast, we leverage both LR and LSTM models for local feature-based training specifically tailored for healthcare data. LR is a straightforward yet effective linear model commonly employed for classification tasks. It operates by using a linear combination of input features to predict the probability of a given example belonging to a particular class. On the other hand, LSTM is a type of RNN well-suited for modeling sequential data, such as time series or natural language. LSTMs excel at capturing long-term dependencies within data by employing gating mechanisms to control information flow throughout the network.

For local feature-based training using LR, we first define a set of local features, compute the local feature representation for each feature vector, and subsequently use these local features as input to train an LR model.

Suppose we have a set of training data, $$(x_1, y_1), (x_2, y_2), \dots, (x_n, y_n)$$, where each $$x_i$$ is a feature vector and $$y_i$$ is the corresponding label. To train a model based on local features, we first need to define a set of local features $$f_1, f_2, \dots, f_m$$ that we will use to represent each feature vector $$x_i$$.

Then, for each feature vector $$x_i$$, we can compute its local feature representation $$z_i$$ as:

Once we have computed the local feature representation for each feature vector in the training data, we can train the LR model using these local features as input and the corresponding labels $$y_i$$ as output.

Before selecting local features, it is important to perform feature engineering and data preprocessing as necessary. Let $$ x_i $$ represent the original feature vector, which consists of a set of local features $$ f_1, f_2, \ldots, f_m $$ that are selected based on domain knowledge and relevance to the problem. In healthcare data analysis, these local features can represent different aspects of patient data or medical measurements. Once the local features are chosen, the local feature representation $$ z_i $$ for each feature vector $$ x_i $$ is computed by applying each local feature function to the original feature vector $$ x_i $$. Here, $$ f_j(x_i) $$ represents the result of applying the $$ j $$-th local feature function to the $$ i $$-th feature vector to train a ML model using these local feature representations $$ z_i $$. The choice of ML algorithms depends on the problem and data characteristics. In our case, we choose both LR and LSTM. To evaluate the model performance using appropriate metrics, we compute accuracy, precision, recall, F1-score, and other relevant metrics based on the model's predictions and the true labels. The relationship between the original feature vector $$x_i$$ and the local feature functions is one of transformation and extraction. The local feature functions transform the raw, high-dimensional data in $$x_i$$ into a set of lower-dimensional features represented by $$f_1(x_i), f_2(x_i), \ldots, f_m(x_i)$$. These local features are chosen based on domain knowledge and relevance to the problem, and they effectively "highlight" certain characteristics of the data while potentially discarding less relevant information. In summary, the local feature functions are used to derive informative features from the original data $$x_i$$, and these features are employed for training ML models in healthcare data analysis. The choice of local features and their calculation methods significantly mold the model's ability to capture relevant patterns in the data.

On the contrary, when employing LSTM for local feature-based training, we adopt a similar process. However, we need to reformat the local feature representation into a sequence suitable for LSTM processing. For the given set of local features $$f_1, f_2, \dots, f_m$$ and each feature vector $$x_i$$, we create a sequence by concatenating these local features. Subsequently, we use this sequence as input to an LSTM model, following the equations described in the Background section.

To replicate our methodology within the healthcare industry, we train the model using the sensitive MIMIC-Ⅲ dataset. Prior to commencing training, we partition the entire dataset into two distinct sets, each containing the same records but with different attributes, as illustrated in Figure 1. During a single training session, we adopt a homogenous label setting, wherein all parties aim to predict the same target variable. However, each party may have access to different features that are relevant to the prediction. To mimic the VFL scenario, we segregate the full dataset into two separate sets with identical records but varying features. The model training is then conducted independently on two different clients to compute the local prediction results for both LR and LSTM. In this simulated scenario, we can consider one party as a Hospital and the other party as a local health insurance provider. After suitable data preprocessing, for the same set of records, the two parties may possess different features. For example, as depicted in Figure 1, the Hospital may have features $$ x_1, x_2, x_3 $$, while the health insurance provider may have features $$ x_1, x_4, x_5 $$. Hence, $$ x_1 $$ is the common feature shared by both parties, while the other two features differ. For benchmarking, we use the tasks defined by Harutyunyan et al. ^[34] as below:

Feature weight mechanism

In ML, feature weights are values learned during the model training process to predict the target variable. These weights indicate the relative importance of each input feature in making predictions. For example, in a model predicting the mortality rate of ICU patients based on features such as length of stay (LOS), drug history, and age, the feature weights reveal which features have the most significant impact on the prediction. For instance, the LOS might be assigned the highest weight, followed by age and drug history. In VFL, a weighted feature algorithm is employed to determine the importance of each feature in the final prediction. Each party trains an LR model locally using its own set of features and labels. This process involves iterative optimization of the weights to minimize the difference between predicted probabilities and actual target values. The learned coefficients of the LR model represent the feature weights. Positive weights indicate a positive relationship with the target variable, while negative weights indicate a negative relationship. The magnitude of the weight reflects the strength of the impact. To protect privacy, only the locally calculated coefficients are shared with the central server, along with a local prediction. As the central server does not perform any training, it solely aggregates the coefficients received from each party and uses them for weighted multiplication to obtain the final score. This approach avoids the need to share raw feature data among parties, preserving privacy. The coefficients obtained from the locally trained LR models are used as the feature weights. By leveraging these feature weights, the central server can aggregate the local predictions effectively without compromising data privacy. This process ensures that the importance of each feature is considered in the final prediction without sharing the raw feature data itself.

Finally, the weight matrix is constructed, as exemplified in Table 4. The weight values in this matrix represent the relative importance of each feature in the classification tasks. For instance, when predicting the mortality of hospital patients, the duration of their stay (In Time and Out Time) has a significantly greater effect on the classification compared to Age or Disease Code. Therefore, In Time and Out Time are assigned higher weights of 85%, while Age and Disease Code are given weights of 25% and 40%, respectively. In contrast, when performing phenotyping for acute diseases, the duration of hospital stay (In Time and Out Time) has a minimal impact, whereas the Disease Code plays a more critical role. Hence, in this scenario, In Time and Out Time are assigned lower weights of 15%, while Age and Disease Code receive higher weights of more than 85%. These weight values reflect the varying degrees of influence each feature has on the different classification tasks.

Aggregator server execution

The local predictions using both LR and LSTM models are computed at each individual client and subsequently sent to the aggregator server for the final aggregation. Once all clients have shared their results, the aggregator server utilizes the weight matrix to calculate the ultimate output. The aggregated score is then optimized by minimizing the loss function through the application of the differential function, as illustrated in Table 3.

For a number of parties $$ P = \{1, 2, ... p\} $$, we can simplify the calculation of the final score as below:

In this framework, we only consider the scenarios of two parties where only the local predictions $$ y(x^p) $$ and the corresponding weight matrix $$ w^p $$ are shared to calculate the final prediction. The aggregation of local predictions is carried out using a continuously differentiable function $$ \delta : \mathbb{R} \rightarrow \mathbb{R} $$, which takes into account the provided weight matrix. In future work, we plan to extend the parties to $$ \ge 2 $$ to showcase the experiment.

It is important to note that the server's role is solely to perform the aggregation based on the local predictions and weight matrix without engaging in any ML classifications. As a result, this framework resembles a parallel or distributed learning algorithm.

For instance, in the case of LR training, the local scores from site 1 and site 2 are 0.67 and 0.75, respectively, as shown in Table 3. Subsequently, after obtaining the average weight coefficients from Table 4 as 0.85 and 0.35, the aggregation is performed, resulting in a calculated score of 0.84 $$ (0.67*0.85 + 0.75*0.35) $$. This value lies between the scores of each local party due to the inclusion of noise addition and weight multiplication during the calculation process.

Privacy mechanism

A central focus in our design is the preservation of local data privacy. To achieve this objective, we implement a privacy-preserving strategy that involves solely transferring local prediction results derived from the local data and features. The decision to refrain from exchanging raw data is driven by the potential risk of exposing sensitive information to unauthorized entities or servers. To mitigate this privacy concern, we employ the widely accepted DP algorithm ^{[12, 30, 35]}, which establishes a standard for safeguarding the privacy of algorithms working with aggregated data. Ensuring the anonymity of feature characteristics is pivotal, and to achieve this, each party introduces Laplacian noise to their local prediction result, denoted as $$ y(x^p) $$ at party $$ p $$. This noise addition guarantees a heightened level of privacy protection against potential malicious servers or parties. Consequently, the shared prediction, with added DP noise, is represented as $$ y(x^p) + \varepsilon $$, where $$ \varepsilon $$ is a variable representing the Laplacian noise introduced to control the level of privacy. The calibration of noise is based on the function's sensitivity, detailed in ^[12]. The noise exhibits an inverse relationship with performance but a direct correlation with data privacy. Introducing more noise increases data obfuscation but enhances privacy. However, this comes at the cost of lower originality in the data, resulting in reduced ML scores. Therefore, our framework seeks to strike a balance, maintaining a middle ground to simultaneously preserve data privacy and optimize performance. We would like to emphasize that DP stands as a well-established method for ensuring privacy, backed by provable privacy boundaries ^[36]. Extensive theoretical and experimental explorations have already been conducted in this area (e.g., ^[37–40]). Consequently, much research has focused on examining how the adoption of DP affects model accuracy (e.g., ^[41–44]) without delving into its defense against specific privacy attacks. Our work aligns with this methodology.

Experimental setup

In our experiments, we used the MIMIC-Ⅲ dataset and followed the benchmark configuration proposed by Harutyunyan et al. ^[34] for processing time series signals from ICU devices. The test set used in the benchmark was also utilized, and we allocated 15% of the remaining data for the validation set. We conducted all four benchmarking tasks described in the methodology section. For the in-hospital mortality task, we considered only patients who were hospitalized in the ICU for at least 48 hours. We removed clinical notes without a linked chart time and any patients without any notes. In the LSTM model, we used 64 hidden units for decompensation and LOS prediction. The phenotyping classification task involved 25 distinct binary classification tasks based on ICU phenotypic data.

Additionally, we tested our approach on another public dataset called Adult, a traditional census dataset with 48, 842 samples, each having 124 characteristics.

The experiments were conducted using both LR and LSTM models in four different scenarios for both datasets. Our test setup consisted of a central server and two clients. In the benchmarking phase, we first examined the prediction in a central architecture where all the datasets with all attributes were available in a single location. Next, we calculated the prediction independently for each local client. Finally, to simulate the VDL architecture, we performed the experiments with two clients and a central server. To create the VDL scenario, we used the Linux Cut command to split the data columns-wise for the identical set of records, effectively separating the entire dataset by feature.

Evaluation

We used the Area Under Precision-Recall (AUC) measure to evaluate the performance of the LR and LSTM models for the in-hospital mortality, phenotyping, and decompensation tasks in the MIMIC dataset. For the LOS task, we used Cohen's linear weighted kappa, as proposed by Harutyunyan et al. ^[34], which assesses the correlation between predicted and actual multi-class buckets. We presented the results in box plots, with red, pink, blue, and yellow boxes representing non-private central, VDL, and local predictions for parties 1 and 2, respectively.

Figure 3(a), Figure 3(b), and Figure 3(c) show the AUC values for the three tasks in the MIMIC dataset. The central architecture consistently performs best since it serves as the benchmark reference with access to the full dataset and all features without privacy constraints. However, the local predictions for parties 1 and 2 have relatively lower scores compared to the central architecture, which is expected as they are trained only on the available local features. On the other hand, the VDL approach outperforms the local parties and comes close to the central architecture's performance. The slight drop in scores for VDL is due to the addition of noise and the weighted feature application during the final score computation on the central server. Figure 3(d) shows the kappa values for the LOS forecasting task. Similar relationships are observed, where the VDL approach performs better than the local predictions but lower than the central architecture. We employed the AUC metric in the same way for the Adult Dataset, and Figure 2 illustrates that the VDL approach achieves higher scores compared to the local parties, approaching the performance of the central architecture.

Figure 2. AUC for Adult Dataset with Privacy Budget $$ \varepsilon $$ : 1 to 5 [RED: Central], [PINK: VDL], [BLUE: Local1], [YELLOW: Local2]

Figure 3. AUC for the MIMIC-Ⅲ Dataset [RED: Central], [PINK: VDL], [BLUE: Local1], [YELLOW: Local2]

Furthermore, we noticed that the LSTM consistently outperforms the LR technique in all figures. This is expected as LSTM benefits from multiple epochs and continuous improvement in deep NN training. When LSTM approaches the LR score, we conclude the epoch rounds to demonstrate LSTM's dominance over LR.

To ensure the confidentiality of the shared score, we implement the epsilon ($$ \varepsilon $$) DP algorithm ^[12]. Epsilon, a pivotal parameter in DP, significantly influences the ML score or model performance. It acts as a decisive factor in determining the degree of privacy protection applied to the data, thereby introducing a trade-off between privacy and utility. As epsilon decreases, the level of privacy protection intensifies, albeit at the expense of utility or the accuracy of the ML model. Lower epsilon values permit the introduction of more noise into the data, enhancing privacy but potentially diminishing the accuracy of the model's predictions—a phenomenon evident in our experiment. Meanwhile, the increase of epsilon values just performs the opposite. In our study, we specifically utilized the epsilon range of 1 to 5. Each box in Figure 3 represents the lower, upper, and median scores of an ML model. The upper value signifies $$ \varepsilon = 5 $$, while the lower part of the box denotes $$ \varepsilon = 1 $$. Consequently, each box encapsulates a privacy-utility spectrum, with the upper portion exhibiting higher utility at the cost of lower privacy and the lower portion featuring lower utility with increased privacy noise. Our experimental results consistently demonstrate that the median converges to a point where an optimal balance between privacy and utility is achieved, effectively avoiding the inherent trade-off. In the realm of DP, Laplacian noise is commonly introduced to the data to fulfill privacy guarantees. The degree of noise injected is modulated by epsilon. Lower epsilon values correspond to increased noise, serving to safeguard privacy but potentially distorting the original data and compromising the model's accuracy. Striking the right balance is pivotal, with the choice of epsilon contingent on specific privacy requirements and the acceptable level of impact on the model's accuracy.

In conclusion, our proposed technique maintains an acceptable level of accuracy for both datasets, even with a minimal privacy budget of 1 to 5. The score disparities observed in comparison to the central server can be attributed to noise addition during local result sharing and the weighted feature computation with loss minimization on the central server. This highlights the trade-off between privacy and utility in this context. However, given that our approach maintains data locality and sensitive patient information remains local, it can be a viable solution in the healthcare sector where privacy preservation is crucial. Our architecture provides a robust privacy-preserving distributed ML framework in this regard.

Acknowledgments

We sincerely thank the reviewers for their insightful comments. This manuscript is based on the dissertation "Privacy-Preserving Federated Learning Model for healthcare data" ^[45].

Authors' contributions

Performed data acquisition and experiments, conducted initial full writing, and provided material support: Islam TU

Made substantial contributions to the conception and revisions of the writing: Mohammed N, Alhadidi D

Availability of data and materials

Not applicable.

Financial support and sponsorship

This research was supported in part by the NSERC Discovery Grants (RGPIN-04127-2022).

Conflicts of interest

All authors declared that there are no conflicts of interest.

Ethical approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Copyright

© The Author(s) 2024.