Zhiyi Zhang, Pengfei Zhang, Qi Wang
Java is very powerful, but in Deep Learning field, its capabilities probably has not been sufficiently exploited. Compared to the Java-based deep-learning-frameworks, the Python-based (PyTorch, TensorFlow, etc) are undoubtedly the mainstream, due to their easy-to-use, flexibility and better ecosystem. Dragon-Alpha is a Java-based Tensor Computing Framework, with easy-to-use, high-scalability and high-performance, trying to break Java's dilemma in deep learning field and make it more effective. Dragon-Alpha supports different levels of APIs, and can be used as a deep-learning-framework through its user-friendly high-level APIs. Dragon-Alpha has potential to aggregate computing-power across heterogeneous platforms and devices, based on its multi-layer architecture and Java's big-data ecosystem. Dragon-Alpha has its asynchronized APIs to improve parallelism, and highly-optimized CUDA library cu32 which adopts unique convolution\deconvolution operators for small feature maps. The experiments show that, compared to PyTorch&cuDNN, Dragon-Alpha&cu32 costs less time and memory (75.38% to 97.32%, 29.2% to 66.4%), to train some typical neural networks (AlexNet, VGG, GoogleNet, ResNet) on Cifar-10.
Zhang Zhiyi, Liu Ziyin
Adaptive gradient methods have achieved remarkable success in training deep neural networks on a wide variety of tasks. However, not much is known about the mathematical and statistical properties of this family of methods. This work aims at providing a series of theoretical analyses of its statistical properties justified by experiments. In particular, we show that when the underlying gradient obeys a normal distribution, the variance of the magnitude of the \textit{update} is an increasing and bounded function of time and does not diverge. This work suggests that the divergence of variance is not the cause of the need for warm up of the Adam optimizer, contrary to what is believed in the current literature.
Zhiyi Zhang, Su Yong Wong, Junxiao Shi, Davide Pesavento, Alexander Afanasyev, Lixia Zhang
Named Data Networking (NDN) secures network communications by requiring all data packets to be signed when produced. This requirement necessitates efficient and usable mechanisms to handle NDN certificate issuance and revocation, making these supporting mechanisms essential for NDN operations. In this paper, we first investigate and clarify core concepts related to NDN certificates and security design in general, and then present the model of NDN certificate management and its desired properties. We proceed with the design of a specific realization of NDN's certificate management, NDNCERT, evaluate it using a formal security analysis, and discuss the challenges in designing, implementing, and deploying the system, to share our experiences with other NDN security protocol development efforts.
Zhiyi Zhang, Vishrant Vasavada, Siva Kesava Reddy Kakarla, Angelos Stavrou, Eric Osterweil, Lixia Zhang
Distributed Denial of Service (DDoS) attacks have plagued the Internet for decades, but the basic defense approaches have not fundamentally changed. Rather, the size and rate of growth in attacks have actually outpaced carriers' and DDoS mitigation services' growth, calling for new solutions that can be, partially or fully, deployed imminently and exhibit effectiveness. In this paper, we examine the basic functions in Named Data Networking (NDN), a newly proposed Internet architecture, that can address the principle weaknesses in today's IP networks. We demonstrate by a new DDoS mitigation solution over NDN, Fine-grained Interest Traffic Throttling FITT, that NDN's architectural changes, even when incrementally deployed, can make DDoS attacks fundamentally more difficult to launch and less effective. FITT leverages the NDN design to enable the network to detect DDoS from victim's feedback, throttles DDoS traffic by reverse its exact paths through the network, and enforces control over the misbehaving entities at their sources. Our extensive simulation results show that FITT can throttle attack traffic with one-way time delay from the victim to the NDN gateway; upon activation, FITT effectively stop attack traffic from impacting benign flows, resulting in over 99\% of packets reaching victims being legitimate ones. We further demonstrate that service providers may implement NDN/FITT on existing CDN nodes as an incrementally deployable solution to effectuate the application level remediation at the sources, which remains unattainable in today's DDoS mitigation approaches.
Zhiyi Zhang
Mutual information is one of the essential building blocks of information theory. Yet, it is only finitely defined for distributions with fast decaying tails on a countable joint alphabet of two random elements. The unboundedness of mutual information over the general class of all distributions on a joint alphabet prevents its potential utility to be fully realized. This is in fact a void in the foundation of information theory that needs to be filled. This article proposes a family of generalized mutual information all of whose members 1) are finitely defined for each and every distribution of two random elements on a joint countable alphabet, except the one by Shannon, and 2) enjoy all utilities of a finite Shannon's mutual information.
Zhiyi Zhang, Yu Guan, Xinyu Ma, Lixia Zhang
As Personally Identifiable Information (PII) data sharing among multiple parties becomes increasingly common, so does the potential for data leakage. As required by new data protection regulations and laws, when PII leakage occurs, one must be able to reliably identify the leaking sources. Existing solutions utilize watermark technologies or data object allocation strategies to differentiate the data shared with different parties to identify potential leakers. However, these solutions lose their effectiveness under several attack scenarios, e.g., a data sender may leak the data and a receiver may deny the reception of certain shared data. Worse yet, multiple receivers might collude and apply a set of operations such as intersection, complement, and union to their received datasets before leaking them, making the task of leaker identification even more difficult. In this paper, we propose AuditShare, a PII dataset sharing system with reliable leaking source identification. Firstly, taking advantage of the intrinsic properties of PII data, AuditShare allocates data objects to individual sharing parties by PII attributes. Secondly, AuditShare obliviously transfers data between the sender and each receiver and uses a Merkle Tree as an immutable record of the sharing. Thirdly, a knowledge-based identification algorithm is proposed to identify a guilty sender or colluding/non-colluding receivers. Through our evaluation, we show that: (i) With a modest amount of leaked data, AuditShare can accurately (accuracy>99.99%) and undeniably identify all the guilty parties in different cases; (ii) It only takes 0.5 second to share 100,000 data objects in AuditShare, which is practical in real-world deployment.
Zhiyi Zhang, Vishrant Vasavada, Xinyu Ma, Lixia Zhang
With the ever growing Internet of Things (IoT) market, ledger systems are facing new challenges to efficiently store and secure enormous customer records collected by the IoT devices. The authenticity, availability, and integrity of these records are critically important for both business providers and customers. In this paper, we describe DLedger, a lightweight and resilient distributed ledger system. Instead of a single chain of blocks, DLedger builds the ledger over a directed acyclic graph (DAG), so that its operations can tolerate network partition and intermittent connectivity. Instead of compute-intensive Proof-of-Work (PoW), DLedger utilizes Proof-of-Authentication (PoA), whose light-weight operations are IoT-friendly, to achieve consensus. Furthermore, DLedger is built upon a data-centric network called Named Data Networking (NDN), which facilitates the peer-to-peer data dissemination in heterogeneous IoT networks.
Zhiyi Zhang, Pengfei Zhang, Zhuopin Xu, Qi Wang
Convolutional neural networks necessitate good algorithms to reduce complexity, and sufficient utilization of parallel processors for acceleration. Within convolutional layers, there are three types of operators: convolution used in forward propagation, deconvolution and dilated-convolution utilized in backward propagation. During the execution of these operators, zeros are typically added to tensors, leading to redundant calculations and unnecessary strain on hardware. To circumvent these inefficiencies, we propose the C-K-S algorithm, accompanied by efficient GPU implementations. C-K-S trims filters to exclude zero-padding. For deconvolution and dilated-convolution, C-K-S transforms sparse tensors into dense tensors, and standardizes the local computational rules to simplify the hardware control. The experimental results demonstrate that C-K-S offers good performance in terms of speed and convergence, surpassing the capabilities of PyTorch and cuDNN in certain scenarios.
Xiaoyuan Xie, Zhiyi Zhang, Tsong Yueh Chen, Yang Liu, Pak-Lok Poon, Baowen Xu
Unsupervised machine learning is the training of an artificial intelligence system using information that is neither classified nor labeled, with a view to modeling the underlying structure or distribution in a dataset. Since unsupervised machine learning systems are widely used in many real-world applications, assessing the appropriateness of these systems and validating their implementations with respect to individual users' requirements and specific application scenarios$\,/\,$contexts are indisputably two important tasks. Such assessment and validation tasks, however, are fairly challenging due to the absence of a priori knowledge of the data. In view of this challenge, we develop a $\textbf{MET}$amorphic $\textbf{T}$esting approach to assessing and validating unsupervised machine $\textbf{LE}$arning systems, abbreviated as METTLE. Our approach provides a new way to unveil the (possibly latent) characteristics of various machine learning systems, by explicitly considering the specific expectations and requirements of these systems from individual users' perspectives. To support METTLE, we have further formulated 11 generic metamorphic relations (MRs), covering users' generally expected characteristics that should be possessed by machine learning systems. To demonstrate the viability and effectiveness of METTLE we have performed an experiment involving six commonly used clustering systems. Our experiment has shown that, guided by user-defined MR-based adequacy criteria, end users are able to assess, validate, and select appropriate clustering systems in accordance with their own specific needs. Our investigation has also yielded insightful understanding and interpretation of the behavior of the machine learning systems from an end-user software engineering's perspective, rather than a designer's or implementor's perspective, who normally adopts a theoretical approach.
Zhiyi Zhang, Tianyuan Yu, Xinyu Ma, Yu Guan, Philipp Moll, Lixia Zhang
Recent years have witnessed the rapid deployment of smart homes; most of them are controlled by remote servers in the cloud. Such designs raise security and privacy concerns for end users. In this paper, we describe the design of Sovereign, a home IoT system framework that provides end users complete control of their home IoT systems. Sovereign lets home IoT devices and applications communicate via application-named data and secures data directly. This enables direct, secure, one-to-one and one-to-many device-to-device communication over wireless broadcast media. Sovereign utilizes semantic names to construct usable security solutions. We implement Sovereign as a publish-subscribe-based development platform together with a prototype home IoT controller. Our preliminary evaluation shows that Sovereign provides a systematic, easy-to-use solution to user-controlled, self-contained smart homes running on existing IoT hardware without imposing noticeable overhead.
Zhiyi Zhang, Siqi Liu, Randy King, Lixia Zhang
Modern digitally controlled systems require multiparty authentication and authorization to meet the desired security requirement. This paper describes the design and development of NDN-MPS, an automated solution to support multiparty signature signing and verification for NDN-enabled applications. NDN-MPS suggests several changes and extensions to the existing NDN security solutions. First, it introduces a new type of trust schema to support signing and verification for multiple signers under complex policies such as threshold schemes. Second, it extends the NDN signature format to accommodate multisignature schemes such as BLS signature. Third, it introduces a signature collection protocol to solicit signatures securely from multiple signers. We further evaluate NDN-MPS by assessing its security properties and measuring its performance.
Zhiyi Zhang, Yingdi Yu, Sanjeev Kaushik Ramani, Alex Afanasyev, Lixia Zhang
In this paper we present the design of Name-based Access Control (NAC) scheme, which supports data confidentiality and access control in Named Data Networking (NDN) architecture by encrypting content at the time of production, and by automating the distribution of encryption and decryption keys. NAC achieves the above design goals by leveraging specially crafted NDN naming conventions to define and enforce access control policies, and to automate the cryptographic key management. The paper also explains how NDN's hierarchically structured namespace allows NAC to support fine-grained access control policies, and how NDN's Interest-Data exchange can help NAC to function in case of intermittent connectivity. Moreover, we show that NAC design can be further extended to support Attribute-based Encryption (ABE), which supports access control with additional levels of flexibility and scalability.
Zhiyi Zhang, Michał Król, Alberto Sonnino, Lixia Zhang, Etienne Rivière
We introduce EL PASSO, a privacy-preserving, asynchronous Single Sign-On (SSO) system. It enables personal authentication while protecting users' privacy against both identity providers and relying parties, and allows selective attribute disclosure. EL PASSO is based on anonymous credentials, yet it supports users' accountability. Selected authorities may recover the identity of allegedly misbehaving users, and users can prove properties about their identity without revealing it in the clear. EL PASSO does not require specific secure hardware or a third party (other than existing participants in SSO). The generation and use of authentication credentials are asynchronous, allowing users to sign on when identity providers are temporarily unavailable. We evaluate EL PASSO in a distributed environment and prove its low computational cost, yielding faster sign-on operations than OIDC from a regular laptop, one-second user-perceived latency from a low-power device, and scaling to more than 50 sign-on operations per second at a relying party using a single 4-core server in the cloud.
Zhiyi Zhang
For each probability distribution on a countable alphabet, a sequence of positive functionals are developed as tail indices based on Turing's perspective. By and only by the asymptotic behavior of these indices, domains of attraction for all probability distributions on the alphabet are defined. The three main domains of attraction are shown to contain distributions with thick tails, thin tails and no tails respectively, resembling in parallel the three main domains of attraction, Gumbel, Frechet and Weibull families, for continuous random variables on the real line. In addition to the probabilistic merits associated with the domains, the tail indices are partially motivated by the fact that there exists an unbiased estimator for every index in the sequence, which is therefore statistically observable, provided that the sample is sufficiently large.
Yingqing He, Zhiyi Zhang, Jiapeng Zhu, Yujun Shen, Qifeng Chen
Understanding the mechanism of generative adversarial networks (GANs) helps us better use GANs for downstream applications. Existing efforts mainly target interpreting unconditional models, leaving it less explored how a conditional GAN learns to render images regarding various categories. This work fills in this gap by investigating how a class conditional generator unifies the synthesis of multiple classes. For this purpose, we dive into the widely used class-conditional batch normalization (CCBN), and observe that each feature channel is activated at varying degrees given different categorical embeddings. To describe such a phenomenon, we propose channel awareness, which quantitatively characterizes how a single channel contributes to the final synthesis. Extensive evaluations and analyses on the BigGAN model pre-trained on ImageNet reveal that only a subset of channels is primarily responsible for the generation of a particular category, similar categories (e.g., cat and dog) usually get related to some same channels, and some channels turn out to share information across all classes. For good measure, our algorithm enables several novel applications with conditional GANs. Concretely, we achieve (1) versatile image editing via simply altering a single channel and manage to (2) harmoniously hybridize two different classes. We further verify that the proposed channel awareness shows promising potential in (3) segmenting the synthesized image and (4) evaluating the category-wise synthesis performance.
Zhihong Chen, Feng Jiang, Junying Chen, Tiannan Wang, Fei Yu, Guiming Chen, Hongbo Zhang, Juhao Liang, Chen Zhang, Zhiyi Zhang, Jianquan Li, Xiang Wan, Benyou Wang, Haizhou Li
This paper presents our efforts to democratize ChatGPT across language. We release a large language model "Phoenix", achieving competitive performance among open-source English and Chinese models while excelling in languages with limited resources (covering both Latin and non-Latin languages). We believe this work will be beneficial to make ChatGPT more accessible, especially in countries where people cannot use ChatGPT due to restrictions from OpenAI or local goverments. Our data, code, and models are available at https://github.com/FreedomIntelligence/LLMZoo.
Xidong Wang, Guiming Hardy Chen, Dingjie Song, Zhiyi Zhang, Zhihong Chen, Qingying Xiao, Feng Jiang, Jianquan Li, Xiang Wan, Benyou Wang, Haizhou Li
Large Language Models (LLMs) provide a possibility to make a great breakthrough in medicine. The establishment of a standardized medical benchmark becomes a fundamental cornerstone to measure progression. However, medical environments in different regions have their local characteristics, e.g., the ubiquity and significance of traditional Chinese medicine within China. Therefore, merely translating English-based medical evaluation may result in \textit{contextual incongruities} to a local region. To solve the issue, we propose a localized medical benchmark called CMB, a Comprehensive Medical Benchmark in Chinese, designed and rooted entirely within the native Chinese linguistic and cultural framework. While traditional Chinese medicine is integral to this evaluation, it does not constitute its entirety. Using this benchmark, we have evaluated several prominent large-scale LLMs, including ChatGPT, GPT-4, dedicated Chinese LLMs, and LLMs specialized in the medical domain. We hope this benchmark provide first-hand experience in existing LLMs for medicine and also facilitate the widespread adoption and enhancement of medical LLMs within China. Our data and code are publicly available at https://github.com/FreedomIntelligence/CMB.
Juhao Liang, Ziwei Wang, Zhuoheng Ma, Jianquan Li, Zhiyi Zhang, Xiangbo Wu, Benyou Wang
Large Language Models(LLMs) have dramatically revolutionized the field of Natural Language Processing(NLP), offering remarkable capabilities that have garnered widespread usage. However, existing interaction paradigms between LLMs and users are constrained by either inflexibility, limitations in customization, or a lack of persistent learning. This inflexibility is particularly evident as users, especially those without programming skills, have restricted avenues to enhance or personalize the model. Existing frameworks further complicate the model training and deployment process due to their computational inefficiencies and lack of user-friendly interfaces. To overcome these challenges, this paper introduces a novel interaction paradigm-'Online Training using External Interactions'-that merges the benefits of persistent, real-time model updates with the flexibility for individual customization through external interactions such as AI agents or online/offline knowledge bases.
Huang Huang, Fei Yu, Jianqing Zhu, Xuening Sun, Hao Cheng, Dingjie Song, Zhihong Chen, Abdulmohsen Alharthi, Bang An, Juncai He, Ziche Liu, Zhiyi Zhang, Junying Chen, Jianquan Li, Benyou Wang, Lian Zhang, Ruoyu Sun, Xiang Wan, Haizhou Li, Jinchao Xu
This paper is devoted to the development of a localized Large Language Model (LLM) specifically for Arabic, a language imbued with unique cultural characteristics inadequately addressed by current mainstream models. Significant concerns emerge when addressing cultural sensitivity and local values. To address this, the paper proposes a comprehensive solution that includes further pre-training with Arabic texts, Supervised Fine-Tuning (SFT) utilizing native Arabic instructions, and GPT-4 responses in Arabic, alongside Reinforcement Learning with AI Feedback (RLAIF) employing a reward model attuned to local culture and values. The goal is to cultivate culturally cognizant and value-aligned Arabic LLMs capable of accommodating the diverse, application-specific needs of Arabic-speaking communities. Comprehensive evaluations reveal that the resulting model, dubbed `AceGPT', sets the state-of-the-art standard for open Arabic LLMs across various benchmarks. Codes, data, and models are in https://github.com/FreedomIntelligence/AceGPT.
Hao Jin, Minghui Lian, Shicheng Qiu, Xuxu Han, Xizhi Zhao, Long Yang, Zhiyi Zhang, Haoran Xie, Kouichi Konno, Shaojun Hu
Creating visually pleasing stylized ink paintings from 3D models is a challenge in robotic manipulation. We propose a semi-automatic framework that can extract expressive strokes from 3D models and draw them in oriental ink painting styles by using a robotic arm. The framework consists of a simulation stage and a robotic drawing stage. In the simulation stage, geometrical contours were automatically extracted from a certain viewpoint and a neural network was employed to create simplified contours. Then, expressive digital strokes were generated after interactive editing according to user's aesthetic understanding. In the robotic drawing stage, an optimization method was presented for drawing smooth and physically consistent strokes to the digital strokes, and two oriental ink painting styles termed as Noutan (shade) and Kasure (scratchiness) were applied to the strokes by robotic control of a brush's translation, dipping and scraping. Unlike existing methods that concentrate on generating paintings from 2D images, our framework has the advantage of rendering stylized ink paintings from 3D models by using a consumer-grade robotic arm. We evaluate the proposed framework by taking 3 standard models and a user-defined model as examples. The results show that our framework is able to draw visually pleasing oriental ink paintings with expressive strokes.