Daniel Appelö, Lu Zhang, Thomas Hagstrom, Fengyan Li
We extend and analyze the energy-based discontinuous Galerkin method for second order wave equations on staggered and structured meshes. By combining spatial staggering with local time-stepping near boundaries, the method overcomes the typical numerical stiffness associated with high order piecewise polynomial approximations. In one space dimension with periodic boundary conditions and suitably chosen numerical fluxes, we prove bounds on the spatial operators that establish stability for CFL numbers $c \frac {Δt}{h} < C$ independent of order when stability-enhanced explicit time-stepping schemes of matching order are used. For problems on bounded domains and in higher dimensions we demonstrate numerically that one can march explicitly with large time steps at high order temporal and spatial accuracy.
Lu Zhang
This paper proposes and analyzes a fully discrete scheme that discretizes space with an ultra-weak local discontinuous Galerkin scheme and time with the Crank--Nicolson method for the nonlinear biharmonic Schrödinger equation. We first rewrite the problem into a system with a second-order spatial derivative and then apply the ultra-weak discontinuous Galerkin method to the system. The proposed scheme is more computationally efficient compared with the local discontinuous Galerkin method because of fewer auxiliary variables, and unconditionally stable without any penalty terms; it also preserves the mass and Hamiltonian conservation that are important properties of the nonlinear biharmonic Schrödinger equation. We also derive optimal L2-error estimates of the semi-discrete scheme that measure both the solution and the auxiliary variable with general nonlinear terms. Several numerical studies demonstrate and support our theoretical findings.
Lu Zhang, Zhiyong Liu, Xiangyu Zhu, Zhan Song, Xu Yang, Zhen Lei, Hong Qiao
To achieve accurate and robust object detection in the real-world scenario, various forms of images are incorporated, such as color, thermal, and depth. However, multimodal data often suffer from the position shift problem, i.e., the image pair is not strictly aligned, making one object has different positions in different modalities. For the deep learning method, this problem makes it difficult to fuse multimodal features and puzzles the convolutional neural network (CNN) training. In this article, we propose a general multimodal detector named aligned region CNN (AR-CNN) to tackle the position shift problem. First, a region feature (RF) alignment module with adjacent similarity constraint is designed to consistently predict the position shift between two modalities and adaptively align the cross-modal RFs. Second, we propose a novel region of interest (RoI) jitter strategy to improve the robustness to unexpected shift patterns. Third, we present a new multimodal feature fusion method that selects the more reliable feature and suppresses the less useful one via feature reweighting. In addition, by locating bounding boxes in both modalities and building their relationships, we provide novel multimodal labeling named KAIST-Paired. Extensive experiments on 2-D and 3-D object detection, RGB-T, and RGB-D datasets demonstrate the effectiveness and robustness of our method.
Lu Zhang, Yang Wang, Jiaogen Zhou, Chenbo Zhang, Yinglu Zhang, Jihong Guan, Yatao Bian, Shuigeng Zhou
Few-shot object detection (FSOD) is to detect objects with a few examples. However, existing FSOD methods do not consider hierarchical fine-grained category structures of objects that exist widely in real life. For example, animals are taxonomically classified into orders, families, genera and species etc. In this paper, we propose and solve a new problem called hierarchical few-shot object detection (Hi-FSOD), which aims to detect objects with hierarchical categories in the FSOD paradigm. To this end, on the one hand, we build the first large-scale and high-quality Hi-FSOD benchmark dataset HiFSOD-Bird, which contains 176,350 wild-bird images falling to 1,432 categories. All the categories are organized into a 4-level taxonomy, consisting of 32 orders, 132 families, 572 genera and 1,432 species. On the other hand, we propose the first Hi-FSOD method HiCLPL, where a hierarchical contrastive learning approach is developed to constrain the feature space so that the feature distribution of objects is consistent with the hierarchical taxonomy and the model's generalization power is strengthened. Meanwhile, a probabilistic loss is designed to enable the child nodes to correct the classification errors of their parent nodes in the taxonomy. Extensive experiments on the benchmark dataset HiFSOD-Bird show that our method HiCLPL outperforms the existing FSOD methods.
Lu Zhang, Ziyang Yuan, Hongxia Wang, Hui Zhang
The randomized sparse Kaczmarz method, designed for seeking the sparse solutions of the linear systems $Ax=b$, selects the $i$-th projection hyperplane with likelihood proportional to $\|a_{i}\|_2^2$, where $a_{i}^T$ is $i$-th row of $A$. In this work, we propose a weighted randomized sparse Kaczmarz method, which selects the $i$-th projection hyperplane with probability proportional to $\lvert\langle a_{i},x_{k}\rangle-b_{i}\rvert^p$, where $0<p<\infty$, for possible acceleration. It bridges the randomized Kaczmarz and greedy Kaczmarz by parameter $p$. Theoretically, we show its linear convergence rate in expectation with respect to the Bregman distance in the noiseless and noisy cases, which is at least as efficient as the randomized sparse Kaczmarz method. The superiority of the proposed method is demonstrated via a group of numerical experiments.
Jiao Chen, Danqing He, Guozhen Lu, Bae Jun Park, Lu Zhang
In this paper, we investigate the Hörmander type theorems for the multi-linear and multi-parameter Fourier multipliers. When the multipliers are characterized by $L^u$-based Sobolev norms for $1<u\le 2$ , our results on the smoothness assumptions are sharp in the multi-parameter and bilinear case. In the multi-parameter and multi-linear case, our results are almost sharp. Moreover, even in the one-parameter and multi-linear case, our results improve earlier ones in the literature.
Lu Zhang, Kam Wai Clifford Chan
Feb 28, 2017·quant-ph·PDF The simultaneous multi-parameter estimation problem using a class of multi-mode entangled states is investigated in this paper. Specifically, the problem of optical phase imaging is considered and the quantum probe is taken to be a balanced coherent superposition of components with an arbitrary quantum state in one mode and vacuum states in all the other modes, which is a generalization of the multi-mode NOON state. The analytical form for the quantum Cramer-Rao bound (QCRB) is presented, which shows the performance by providing a lower bound of the estimation uncertainty. It is shown that the NOON state has the worst performance among those in the class of the entangled states considered. We also analyze in detail four different scenarios, which are the NOON state, the entangled coherent state, the entangled squeezed coherent state, and the entangled squeezed vacuum state. From the comparison among these four states, we find that when the mean photon number is fixed, the squeezed vacuum state has the smallest QCRB, followed by the squeezed coherent state, entangled coherent state, and NOON state. We also illustrate that the balanced entangled state can perform better than a more generalized unbalanced form studied in previous works for certain scenarios. Finally, we give an experimental setup for producing a two-mode entangled state that can beat the NOON state in quantum metrology.
Lu Zhang, Yongkai Wu, Xintao Wu
Discrimination discovery and prevention/removal are increasingly important tasks in data mining. Discrimination discovery aims to unveil discriminatory practices on the protected attribute (e.g., gender) by analyzing the dataset of historical decision records, and discrimination prevention aims to remove discrimination by modifying the biased data before conducting predictive analysis. In this paper, we show that the key to discrimination discovery and prevention is to find the meaningful partitions that can be used to provide quantitative evidences for the judgment of discrimination. With the support of the causal graph, we present a graphical condition for identifying a meaningful partition. Based on that, we develop a simple criterion for the claim of non-discrimination, and propose discrimination removal algorithms which accurately remove discrimination while retaining good data utility. Experiments using real datasets show the effectiveness of our approaches.
Lu Zhang, Abhirup Datta, Sudipto Banerjee
With continued advances in Geographic Information Systems and related computational technologies, statisticians are often required to analyze very large spatial datasets. This has generated substantial interest over the last decade, already too vast to be summarized here, in scalable methodologies for analyzing large spatial datasets. Scalable spatial process models have been found especially attractive due to their richness and flexibility and, particularly so in the Bayesian paradigm, due to their presence in hierarchical model settings. However, the vast majority of research articles present in this domain have been geared toward innovative theory or more complex model development. Very limited attention has been accorded to approaches for easily implementable scalable hierarchical models for the practicing scientist or spatial analyst. This article is submitted to the Practice section of the journal with the aim of developing massively scalable Bayesian approaches that can rapidly deliver Bayesian inference on spatial process that are practically indistinguishable from inference obtained using more expensive alternatives. A key emphasis is on implementation within very standard (modest) computing environments (e.g., a standard desktop or laptop) using easily available statistical software packages without requiring message-parsing interfaces or parallel programming paradigms. Key insights are offered regarding assumptions and approximations concerning practical efficiency.
Lu Zhang, Andrew O. Finley, Arne Nothdurft, Sudipto Banerjee
Modeling incompatible spatial data, i.e., data with different spatial resolutions, is a pervasive challenge in remote sensing data analysis. Typical approaches to addressing this challenge aggregate information to a common coarse resolution, i.e., compatible resolutions, prior to modeling. Such pre-processing aggregation simplifies analysis, but potentially causes information loss and hence compromised inference and predictive performance. To avoid losing potential information provided by finer spatial resolution data and improve predictive performance, we propose a new Bayesian method that constructs a latent spatial process model at the finest spatial resolution. This model is tailored to settings where the outcome variable is measured on a coarser spatial resolution than predictor variables -- a configuration seen increasingly when high spatial resolution remotely sensed predictors are used in analysis. A key contribution of this work is an efficient algorithm that enables full Bayesian inference using finer resolution data while optimizing computational and storage costs. The proposed method is applied to a forest damage assessment for the 2018 Adrian storm in Carinthia, Austria, that uses high-resolution laser imaging detection and ranging (LiDAR) measurements and relatively coarse resolution forest inventory measurements. Extensive simulation studies demonstrate the proposed approach substantially improves inference for small prediction units.
Lu Zhang, Liang Zeng
The rapid proliferation of AI-generated images necessitates effective watermarking techniques to protect intellectual property and detect fraudulent content. While existing training-based watermarking methods show promise, they often struggle with generalizing across diverse prompts and tend to introduce visible artifacts. To this end, we propose a novel, provably generalizable image watermarking approach for Latent Diffusion Models, termed Self-Augmented Training (SAT-LDM). Our method aligns the training and testing phases through a free generation distribution, thereby enhancing the watermarking module's generalization capabilities. We theoretically consolidate SAT-LDM by proving that the free generation distribution contributes to its tight generalization bound, without the need for additional data collection. Extensive experiments show that SAT-LDM not only achieves robust watermarking but also significantly improves the quality of watermarked images across a wide range of prompts. Moreover, our experimental analyses confirm the strong generalization abilities of SAT-LDM. We hope that our method provides a practical and efficient solution for securing high-fidelity AI-generated content.
Lu Zhang, Sangarapillai Lambotharan, Gan Zheng, Guisheng Liao, Basil AsSadhan, Fabio Roli
Due to great success of transformers in many applications such as natural language processing and computer vision, transformers have been successfully applied in automatic modulation classification. We have shown that transformer-based radio signal classification is vulnerable to imperceptible and carefully crafted attacks called adversarial examples. Therefore, we propose a defense system against adversarial examples in transformer-based modulation classifications. Considering the need for computationally efficient architecture particularly for Internet of Things (IoT)-based applications or operation of devices in environment where power supply is limited, we propose a compact transformer for modulation classification. The advantages of robust training such as adversarial training in transformers may not be attainable in compact transformers. By demonstrating this, we propose a novel compact transformer that can enhance robustness in the presence of adversarial attacks. The new method is aimed at transferring the adversarial attention map from the robustly trained large transformer to a compact transformer. The proposed method outperforms the state-of-the-art techniques for the considered white-box scenarios including fast gradient method and projected gradient descent attacks. We have provided reasoning of the underlying working mechanisms and investigated the transferability of the adversarial examples between different architectures. The proposed method has the potential to protect the transformer from the transferability of adversarial examples.
Lu Zhang, Jinchuan Zeng, Hui Zhang
Large-scale linear systems of the form $Ax=b$ are often doubly-noisy, in the sense that both its measurement matrix $A$ and measurement vector $b$ are noisy. In this paper, we extend the relaxed greedy randomized Kaczmarz (RGRK) method to the doubly-noisy systems to accelerate convergence. However, RGRK fails to converge to the least-squares solution for doubly-noisy systems. To address this limitation, we propose a simple modification: averaging multiple measurements instead of using a single measurement. The proposed RGRK with signal averaging (RGRK-SA) converges to the solution of doubly-noisy systems at a polynomial rate. Numerical experiments demonstrate that both RGRK and RGRK-SA outperform the classical randomized Kaczmarz method, and RGRK-SA has a higher accuracy.
Lu Zhang, Hongzhen Chen, Hongxia Wang, Hui Zhang
Stochastic projection algorithms for solving convex feasibility problems (CFPs) have attracted considerable attention due to their broad applicability. In this paper, we propose a unified stochastic bilevel reformulation for possibly inconsistent CFPs that combines proximity function minimization and structural regularization, leading to a feasible bilevel model with a unique and stable regularized solution. From the algorithmic perspective, we develop the stochastic block Bregman projection method with Polyak-like and projective stepsizes, which not only subsumes several recent stochastic projection algorithms but also induces new schemes tailored to specific problems. Moreover, we establish ergodic sublinear convergence rates for the expected inner function, as well as linear convergence in expectation to the inner minimizer set under a Bregman distance growth condition. In particular, the proposed Polyak-like stepsize ensures exact convergence in expectation for possibly inconsistent CFPs. Finally, numerical experiments demonstrate the effectiveness of the proposed method and its robustness to noise.
Lu Zhang, Siyang Wang, N. Anders Petersson
We develop a fourth order accurate finite difference method for the three dimensional elastic wave equation in isotropic media with the piecewise smooth material property. In our model, the material property can be discontinuous at curved interfaces. The governing equations are discretized in second order form on curvilinear meshes by using a fourth order finite difference operator satisfying a summation-by-parts property. The method is energy stable and high order accurate. The highlight is that mesh sizes can be chosen according to the velocity structure of the material so that computational efficiency is improved. At the mesh refinement interfaces with hanging nodes, physical interface conditions are imposed by using ghost points and interpolation. With a fourth order predictor-corrector time integrator, the fully discrete scheme is energy conserving. Numerical experiments are presented to verify the fourth order convergence rate and the energy conserving property.
Lu Zhang, Chenxing Li, Feng Deng, Xiaorui Wang
The audio source separation tasks, such as speech enhancement, speech separation, and music source separation, have achieved impressive performance in recent studies. The powerful modeling capabilities of deep neural networks give us hope for more challenging tasks. This paper launches a new multi-task audio source separation (MTASS) challenge to separate the speech, music, and noise signals from the monaural mixture. First, we introduce the details of this task and generate a dataset of mixtures containing speech, music, and background noises. Then, we propose an MTASS model in the complex domain to fully utilize the differences in spectral characteristics of the three audio signals. In detail, the proposed model follows a two-stage pipeline, which separates the three types of audio signals and then performs signal compensation separately. After comparing different training targets, the complex ratio mask is selected as a more suitable target for the MTASS. The experimental results also indicate that the residual signal compensation module helps to recover the signals further. The proposed model shows significant advantages in separation performance over several well-known separation models.
Lu Zhang, Sudipto Banerjee, Andrew O. Finley
Joint modeling of spatially-oriented dependent variables is commonplace in the environmental sciences, where scientists seek to estimate the relationships among a set of environmental outcomes accounting for dependence among these outcomes and the spatial dependence for each outcome. Such modeling is now sought for massive data sets with variables measured at a very large number of locations. Bayesian inference, while attractive for accommodating uncertainties through hierarchical structures, can become computationally onerous for modeling massive spatial data sets because of its reliance on iterative estimation algorithms. This manuscript develops a conjugate Bayesian framework for analyzing multivariate spatial data using analytically tractable posterior distributions that obviate iterative algorithms. We discuss differences between modeling the multivariate response itself as a spatial process and that of modeling a latent process in a hierarchical model. We illustrate the computational and inferential benefits of these models using simulation studies and analysis of a Vegetation Index data set with spatially dependent observations numbering in the millions.
Lu Zhang, Wenpin Tang, Sudipto Banerjee
Statistical modeling for massive spatial data sets has generated a substantial literature on scalable spatial processes based upon Vecchia's approximation. Vecchia's approximation for Gaussian process models enables fast evaluation of the likelihood by restricting dependencies at a location to its neighbors. We establish inferential properties of microergodic spatial covariance parameters within the paradigm of fixed-domain asymptotics when they are estimated using Vecchia's approximation. The conditions required to formally establish these properties are explored, theoretically and empirically, and the effectiveness of Vecchia's approximation is further corroborated from the standpoint of fixed-domain asymptotics.
Lu Zhang
Conjugate gradient is an efficient algorithm for solving large sparse linear systems. It has been utilized to accelerate the computation in Bayesian analysis for many large-scale problems. This article discusses the applications of conjugate gradient in Bayesian computation, with a focus on sparse regression and spatial analysis. A self-contained introduction of conjugate gradient is provided to facilitate potential applications in a broader range of problems.
Lu Zhang
Consider a random matrix of size $N$ as an additive deformation of the complex Ginibre ensemble under a deterministic matrix $X_0$ with a finite rank, independent of $N$. We prove that microscopic statistics for the mean diagonal overlap, near the edge point, are characterized by the iterative erfc integrals, which only depend on the geometric multiplicity of certain eigenvalue of $X_0$. We also investigate the microscopic statistics for the mean diagonal overlap of the outlier eigenvalues. Further we get a phenomenon of the phase transition for the mean diagonal overlap, with respect to the modulus of the eigenvalues of $X_0$.