Heterogeneity-aware and communication-efficient distributed statistical inference Article Swipe

MAKE IMPACT

Rui Duan , Yang Ning , Yong Chen · YOU? · · 2019 · Open Access · · DOI: https://doi.org/10.48550/arxiv.1912.09623

In multicenter research, individual-level data are often protected against sharing across sites. To overcome the barrier of data sharing, many distributed algorithms, which only require sharing aggregated information, have been developed. The existing distributed algorithms usually assume the data are homogeneously distributed across sites. This assumption ignores the important fact that the data collected at different sites may come from various sub-populations and environments, which can lead to heterogeneity in the distribution of the data. Ignoring the heterogeneity may lead to erroneous statistical inference. In this paper, we propose distributed algorithms which account for the heterogeneous distributions by allowing site-specific nuisance parameters. The proposed methods extend the surrogate likelihood approach to the heterogeneous setting by applying a novel density ratio tilting method to the efficient score function. The proposed algorithms maintain the same communication cost as the existing communication-efficient algorithms. We establish a non-asymptotic risk bound for the proposed distributed estimator and its limiting distribution in the two-index asymptotic setting which allows both sample size per site and the number of sites to go to infinity. In addition, we show that the asymptotic variance of the estimator attains the Cramér-Rao lower bound when the number of sites is in rate smaller than the sample size at each site. Finally, we use simulation studies and a real data application to demonstrate the validity and feasibility of the proposed methods.

Type

preprint

Language

en

Landing Page

http://arxiv.org/abs/1912.09623

PDF

https://arxiv.org/pdf/1912.09623

OA Status

green

Cited By

9

References

30

Related Works

10

OpenAlex ID

https://openalex.org/W2996417372

Raw OpenAlex JSON

OpenAlex ID

https://openalex.org/W2996417372

Canonical identifier for this work in OpenAlex

DOI

https://doi.org/10.48550/arxiv.1912.09623

Digital Object Identifier

Title

Heterogeneity-aware and communication-efficient distributed statistical inference

Work title

Type

preprint

OpenAlex work type

Language

en

Primary language

Publication year

2019

Year of publication

Publication date

2019-12-20

Full publication date if available

Authors

Rui Duan, Yang Ning, Yong Chen

List of authors in order

Landing page

https://arxiv.org/abs/1912.09623

Publisher landing page

PDF URL

https://arxiv.org/pdf/1912.09623

Direct link to full text PDF

Open access

Yes

Whether a free full text is available

OA status

green

Open access status per OpenAlex

OA URL

https://arxiv.org/pdf/1912.09623

Direct OA link when available

Concepts

Estimator, Computer science, Inference, Statistical inference, Variance (accounting), Sample size determination, Asymptotic distribution, Upper and lower bounds, Function (biology), Likelihood function, Algorithm, Data mining, Statistics, Mathematics, Estimation theory, Artificial intelligence, Biology, Mathematical analysis, Evolutionary biology, Business, Accounting

Top concepts (fields/topics) attached by OpenAlex

Cited by

9

Total citation count in OpenAlex

Citations by year (recent)

2024: 2, 2023: 4, 2021: 1, 2020: 2

Per-year citation counts (last 5 years)

References (count)

30

Number of works referenced by this work

Related works (count)

10

Other works algorithmically related by OpenAlex

Full payload