65 Hits in 1.4 sec

Quantum Neural Network Compression [article]

Zhirui Hu, Peiyan Dong, Zhepeng Wang, Youzuo Lin, Yanzhi Wang, Weiwen Jiang
2022 arXiv   pre-print
Model compression, such as pruning and quantization, has been widely applied to optimize neural networks on resource-limited classical devices. Recently, there are growing interest in variational quantum circuits (VQC), that is, a type of neural network on quantum computers (a.k.a., quantum neural networks). It is well known that the near-term quantum devices have high noise and limited resources (i.e., quantum bits, qubits); yet, how to compress quantum neural networks has not been thoroughly
more » ... tudied. One might think it is straightforward to apply the classical compression techniques to quantum scenarios. However, this paper reveals that there exist differences between the compression of quantum and classical neural networks. Based on our observations, we claim that the compilation/traspilation has to be involved in the compression process. On top of this, we propose the very first systematical framework, namely CompVQC, to compress quantum neural networks (QNNs).In CompVQC, the key component is a novel compression algorithm, which is based on the alternating direction method of multipliers (ADMM) approach. Experiments demonstrate the advantage of the CompVQC, reducing the circuit depth (almost over 2.5 %) with a negligible accuracy drop (<1%), which outperforms other competitors. Another promising truth is our CompVQC can indeed promote the robustness of the QNN on the near-term noisy quantum devices.
arXiv:2207.01578v2 fatcat:4pnkbvid4fczzl6mlbh27nht24

Distributed Contrastive Learning for Medical Image Segmentation [article]

Yawen Wu, Dewen Zeng, Zhepeng Wang, Yiyu Shi, Jingtong Hu
2022 arXiv   pre-print
Supervised deep learning needs a large amount of labeled data to achieve high performance. However, in medical imaging analysis, each site may only have a limited amount of data and labels, which makes learning ineffective. Federated learning (FL) can learn a shared model from decentralized data. But traditional FL requires fully-labeled data for training, which is very expensive to obtain. Self-supervised contrastive learning (CL) can learn from unlabeled data for pre-training, followed by
more » ... -tuning with limited annotations. However, when adopting CL in FL, the limited data diversity on each site makes federated contrastive learning (FCL) ineffective. In this work, we propose two federated self-supervised learning frameworks for volumetric medical image segmentation with limited annotations. The first one features high accuracy and fits high-performance servers with high-speed connections. The second one features lower communication costs, suitable for mobile devices. In the first framework, features are exchanged during FCL to provide diverse contrastive data to each site for effective local CL while keeping raw data private. Global structural matching aligns local and remote features for a unified feature space among different sites. In the second framework, to reduce the communication cost for feature exchanging, we propose an optimized method FCLOpt that does not rely on negative samples. To reduce the communications of model download, we propose the predictive target network update (PTNU) that predicts the parameters of the target network. Based on PTNU, we propose the distance prediction (DP) to remove most of the uploads of the target network. Experiments on a cardiac MRI dataset show the proposed two frameworks substantially improve the segmentation and generalization performance compared with state-of-the-art techniques.
arXiv:2208.03808v1 fatcat:3ruylvqxkfbzbo7db5oiafsupa

Effect of 3-Mercaptopropyltriethoxysilane Modified Illite on the Reinforcement of SBR

Zhepeng Wang, Hao Zhang, Qiang Liu, Shaojuan Wang, Shouke Yan
2022 Materials  
To achieve the sustainable development of the rubber industry, the substitute of carbon black, the most widely used but non-renewable filler produced from petroleum, has been considered one of the most effective ways. The naturally occurring illite with higher aspect ratio can be easily obtained in large amounts at lower cost and with lower energy consumption. Therefore, the expansion of its application in advanced materials is of great significance. To explore their potential use as an
more » ... for reinforcing rubber, styrene butadiene rubber (SBR) composites with illites of different size with and without 3-mercaptopropyltriethoxysilane (KH580) modification were studied. It was found that the modification of illite by KH580 increases the K-illite/SBR interaction, and thus improves the dispersion of K-illite in the SBR matrix. The better dispersion of smaller size K-illite with stronger interfacial interaction improves the mechanical properties of SBR remarkably, by an increment of about nine times the tensile strength and more than ten times the modulus. These results demonstrate, except for the evident effect of particle size, the great importance of filler–rubber interaction on the performance of SBR composites. This may be of great significance for the potential wide use of the abundant naturally occurring illite as substitute filler for the rubber industry.
doi:10.3390/ma15103459 pmid:35629487 pmcid:PMC9143291 fatcat:cy357myouvfarbtp7vssdoo5ae

Decentralized Unsupervised Learning of Visual Representations [article]

Yawen Wu, Zhepeng Wang, Dewen Zeng, Meng Li, Yiyu Shi, Jingtong Hu
2022 arXiv   pre-print
Collaborative learning enables distributed clients to learn a shared model for prediction while keeping the training data local on each client. However, existing collaborative learning methods require fully-labeled data for training, which is inconvenient or sometimes infeasible to obtain due to the high labeling cost and the requirement of expertise. The lack of labels makes collaborative learning impractical in many realistic settings. Self-supervised learning can address this challenge by
more » ... rning from unlabeled data. Contrastive learning (CL), a self-supervised learning approach, can effectively learn visual representations from unlabeled image data. However, the distributed data collected on clients are usually not independent and identically distributed (non-IID) among clients, and each client may only have few classes of data, which degrades the performance of CL and learned representations. To tackle this problem, we propose a collaborative contrastive learning framework consisting of two approaches: feature fusion and neighborhood matching, by which a unified feature space among clients is learned for better data representations. Feature fusion provides remote features as accurate contrastive information to each client for better local learning. Neighborhood matching further aligns each client's local features to the remote features such that well-clustered features among clients can be learned. Extensive experiments show the effectiveness of the proposed framework. It outperforms other methods by 11% on IID data and matches the performance of centralized learning.
arXiv:2111.10763v2 fatcat:tqayp2fl55aylgvfmatgl5tvyi

Personalized Deep Learning for Ventricular Arrhythmias Detection on Medical IoT Systems [article]

Zhenge Jia, Zhepeng Wang, Feng Hong, Lichuan Ping, Yiyu Shi, Jingtong Hu
2020 arXiv   pre-print
Life-threatening ventricular arrhythmias (VA) are the leading cause of sudden cardiac death (SCD), which is the most significant cause of natural death in the US. The implantable cardioverter defibrillator (ICD) is a small device implanted to patients under high risk of SCD as a preventive treatment. The ICD continuously monitors the intracardiac rhythm and delivers shock when detecting the life-threatening VA. Traditional methods detect VA by setting criteria on the detected rhythm. However,
more » ... ose methods suffer from a high inappropriate shock rate and require a regular follow-up to optimize criteria parameters for each ICD recipient. To ameliorate the challenges, we propose the personalized computing framework for deep learning based VA detection on medical IoT systems. The system consists of intracardiac and surface rhythm monitors, and the cloud platform for data uploading, diagnosis, and CNN model personalization. We equip the system with real-time inference on both intracardiac and surface rhythm monitors. To improve the detection accuracy, we enable the monitors to detect VA collaboratively by proposing the cooperative inference. We also introduce the CNN personalization for each patient based on the computing framework to tackle the unlabeled and limited rhythm data problem. When compared with the traditional detection algorithm, the proposed method achieves comparable accuracy on VA rhythm detection and 6.6% reduction in inappropriate shock rate, while the average inference latency is kept at 71ms.
arXiv:2008.08060v1 fatcat:5e65uxuktnba7ei5t73y5m2npq

Enabling On-Device CNN Training by Self-Supervised Instance Filtering and Error Map Pruning [article]

Yawen Wu, Zhepeng Wang, Yiyu Shi, Jingtong Hu
2020 arXiv   pre-print
This work aims to enable on-device training of convolutional neural networks (CNNs) by reducing the computation cost at training time. CNN models are usually trained on high-performance computers and only the trained models are deployed to edge devices. But the statically trained model cannot adapt dynamically in a real environment and may result in low accuracy for new inputs. On-device training by learning from the real-world data after deployment can greatly improve accuracy. However, the
more » ... h computation cost makes training prohibitive for resource-constrained devices. To tackle this problem, we explore the computational redundancies in training and reduce the computation cost by two complementary approaches: self-supervised early instance filtering on data level and error map pruning on the algorithm level. The early instance filter selects important instances from the input stream to train the network and drops trivial ones. The error map pruning further prunes out insignificant computations when training with the selected instances. Extensive experiments show that the computation cost is substantially reduced without any or with marginal accuracy loss. For example, when training ResNet-110 on CIFAR-10, we achieve 68% computation saving while preserving full accuracy and 75% computation saving with a marginal accuracy loss of 1.3%. Aggressive computation saving of 96% is achieved with less than 0.1% accuracy loss when quantization is integrated into the proposed approaches. Besides, when training LeNet on MNIST, we save 79% computation while boosting accuracy by 0.2%.
arXiv:2007.03213v1 fatcat:ecbaiqlalbcy3j6tznrkrksd54

Compound Capecitabine Colon-Targeted Microparticle Prepared by Coaxial Electrospray for Treatment of Colon Tumors

Ruiqi Chen, Ruidong Zhai, Chao Wang, Shulong Liang, Jing Wang, Zhepeng Liu, Wenlin Li
2022 Molecules  
To improve the antitumor effect of combined capecitabine (CAP) and osimertinib (OSI) therapy and quickly and efficiently reduce tumor volumes for preoperative chemotherapy, we designed a compound CAP colon-targeted microparticle (COPMP) prepared by coaxial electrospray. COPMP is a core–shell microparticle composed of a Eudragit S100 outer layer and a CAP/OSI-loaded PLGA core. In this study, we characterized its size distribution, drug loading (DL), encapsulation efficiency (EE), differential
more » ... nning calorimetry (DSC), Fourier transform infrared spectra (FTIR), in vitro release, formula ratio, cellular growth inhibition, and in vivo antitumor efficacy. COPMP is of spherical appearance with a size of 1.87 ± 0.23 μm. The DLs of CAP and OSI are 4.93% and 4.95%, respectively. The DSC showed that the phase state of CAP and OSI changed after encapsulation. The FTIR results indicated good compatibility between the drug and excipients. The release curve showed that CAP and OSI were released in a certain ratio. They were barely released prior to 2 h (pH 1.0), less than 50% was released between 3 and 5 h (pH 6.8), and sustained release of up to 80% occurred between 6 and 48 h (pH 7.4). CAP and OSI demonstrated a synergistic effect on HCT-116 cells. In a colon tumor model, the tumor inhibition rate after oral administration of COPMP reached 94% within one week. All the data suggested that COPMP promotes the sustained release of CAP and OSI in the colon, which provides a preoperative chemotherapy scheme for the treatment of colon cancer.
doi:10.3390/molecules27175690 pmid:36080457 pmcid:PMC9457672 fatcat:c7vcsx5z7vgfhc2bqvphjnuyja

Comparison of HMOX1 expression and enzyme activity in blue-shelled chickens and brown-shelled chickens

ZhePeng Wang, RuiFang Liu, AnRu Wang
2013 Genetics and Molecular Biology  
et al., 2010 (Wang et al., , 2011 and would result in blue-colored eggs.  ...  Previous studies of HMOX1 expression in shell gland suggested that the blue egg coloring was associated with high gene expression in the shell gland of blue-shelled chickens (Wang et al., 2010 (Wang et  ... 
doi:10.1590/s1415-47572013000200020 pmid:23885212 pmcid:PMC3715296 fatcat:f2nrxmbeija5rkpol6apohstke

Federated Contrastive Learning for Dermatological Disease Diagnosis via On-device Learning [article]

Yawen Wu, Dewen Zeng, Zhepeng Wang, Yi Sheng, Lei Yang, Alaina J. James, Yiyu Shi, Jingtong Hu
2022 arXiv   pre-print
Deep learning models have been deployed in an increasing number of edge and mobile devices to provide healthcare. These models rely on training with a tremendous amount of labeled data to achieve high accuracy. However, for medical applications such as dermatological disease diagnosis, the private data collected by mobile dermatology assistants exist on distributed mobile devices of patients, and each device only has a limited amount of data. Directly learning from limited data greatly
more » ... tes the performance of learned models. Federated learning (FL) can train models by using data distributed on devices while keeping the data local for privacy. Existing works on FL assume all the data have ground-truth labels. However, medical data often comes without any accompanying labels since labeling requires expertise and results in prohibitively high labor costs. The recently developed self-supervised learning approach, contrastive learning (CL), can leverage the unlabeled data to pre-train a model, after which the model is fine-tuned on limited labeled data for dermatological disease diagnosis. However, simply combining CL with FL as federated contrastive learning (FCL) will result in ineffective learning since CL requires diverse data for learning but each device only has limited data. In this work, we propose an on-device FCL framework for dermatological disease diagnosis with limited labels. Features are shared in the FCL pre-training process to provide diverse and accurate contrastive information. After that, the pre-trained model is fine-tuned with local labeled data independently on each device or collaboratively with supervised federated learning on all devices. Experiments on dermatological disease datasets show that the proposed framework effectively improves the recall and precision of dermatological disease diagnosis compared with state-of-the-art methods.
arXiv:2202.07470v1 fatcat:uqakmyrsgzfx3j5sozdwd5jhwq

Federated Self-Supervised Contrastive Learning and Masked Autoencoder for Dermatological Disease Diagnosis [article]

Yawen Wu, Dewen Zeng, Zhepeng Wang, Yi Sheng, Lei Yang, Alaina J. James, Yiyu Shi, Jingtong Hu
2022 arXiv   pre-print
In dermatological disease diagnosis, the private data collected by mobile dermatology assistants exist on distributed mobile devices of patients. Federated learning (FL) can use decentralized data to train models while keeping data local. Existing FL methods assume all the data have labels. However, medical data often comes without full labels due to high labeling costs. Self-supervised learning (SSL) methods, contrastive learning (CL) and masked autoencoders (MAE), can leverage the unlabeled
more » ... ta to pre-train models, followed by fine-tuning with limited labels. However, combining SSL and FL has unique challenges. For example, CL requires diverse data but each device only has limited data. For MAE, while Vision Transformer (ViT) based MAE has higher accuracy over CNNs in centralized learning, MAE's performance in FL with unlabeled data has not been investigated. Besides, the ViT synchronization between the server and clients is different from traditional CNNs. Therefore, special synchronization methods need to be designed. In this work, we propose two federated self-supervised learning frameworks for dermatological disease diagnosis with limited labels. The first one features lower computation costs, suitable for mobile devices. The second one features high accuracy and fits high-performance servers. Based on CL, we proposed federated contrastive learning with feature sharing (FedCLF). Features are shared for diverse contrastive information without sharing raw data for privacy. Based on MAE, we proposed FedMAE. Knowledge split separates the global and local knowledge learned from each client. Only global knowledge is aggregated for higher generalization performance. Experiments on dermatological disease datasets show superior accuracy of the proposed frameworks over state-of-the-arts.
arXiv:2208.11278v1 fatcat:cluksc4qybdmjfyovcxjmicnfe

Bivalent Copper Ions Promote Fibrillar Aggregation of KCTD1 and Induce Cytotoxicity

Zhepeng Liu, Feifei Song, Zhi-li Ma, Qiushuang Xiong, Jingwen Wang, Deyin Guo, Guihong Sun
2016 Scientific Reports  
Potassium channel tetramerization domain containing 1 (KCTD1) family members have a BTB/ POZ domain, which can facilitate protein-protein interactions involved in the regulation of different signaling pathways. KCTD proteins have potential Zn 2+ /Cu 2+ binding sites with currently unknown structural and functional roles. We investigated potential Cu 2+ -specific effects on KCTD1 using circular dichroism, turbidity measurement, fluorescent dye binding, proteinase K (PK) digestion, cell
more » ... ion and apoptosis assays. These experiments indicate that the KCTD1 secondary structure assumes greater β-sheet content and the proteins aggregate into a PK-resistant form under 20 μM Cu 2+ , and this β-sheet-rich aggregation with Cu 2+ promotes fibril formation, which results in increased cell toxicity by apoptosis. Our results reveal a novel role for Cu 2+ in determining the structure and function of KCTD1. Metal ions have essential roles in human physiology 1,2 , and many proteins have greater binding affinity for Cu 2+ than other divalent metal ions 3 . Copper has a wide intracellular distribution and essential biofunctions, but it is toxic when in excess. Copper is the third-most abundant metal in brain tissue 4 . Copper levels accumulate in diseased human tissues and are associated with plaque formation of prions, Alzheimer's, Parkinson's, and Huntington's 5-7 . These human diseases have been linked to protein misfolding and aggregation 8-10 . Copper acts as an important cofactor in changing protein secondary structure, and can accelerate or delay protein misfolding and aggregation rate in vitro 6,11 . Copper has been linked to the progression of protein-misfolding diseases 7,12 , which are characterized by the presence of fibrillar aggregates or amyloid plaques. Several studies have suggested that formation of amyloid deposits in neurological disorders results from increased levels of metal ions and subsequent failure of the ubiquitin-proteasomal system (UPS) of protein degradation 13, 14 . Biological investigations have demonstrated that exposure of the UPS to increasing amounts of metal ions affects its primary proteolytic activity, which suggests a close relationship between age-dependent increase in metal ion levels in the brain, UPS failure, and disease onset. Protein ubiquitination is usually mediated by diverse E3 ligase enzymes that serve as scaffolds for interactions with substrate-recruiting adaptor proteins. BTB-domain-containing KCTD proteins may act as adaptors for interaction between Cul3 ubiquitin ligase and its substrates. Recent work shows that KCTD proteins participate as substrate-specific adaptors in multimeric cullin-E3 ligase reactions by recruiting proteins for ubiquitination and subsequent degradation by the proteasome 15-17 . KCTD1 is a member of the KCTD protein family; it contains a bric-a-brac, tramtrack, and broad complex (BTB) domain. Recent studies of KCTD involvement in protein degradation yielded a surprisingly unifying view: KCTD1 enhances the ubiquitination/degradation of β -catenin in a concentration-dependent manner in HeLa cells 18 . Our previous study showed that KCTD1 interacted with the cellular prion protein (PrP C ) with high affinity, and suggested that KCTD1 could serve as an adaptor that mediates PrP ubiquitination and degradation via cullin binding 19 . Copper accumulates in plaques in neurological disorders, and KCTD1 is strongly expressed in brain cells 20 . These results raise some interesting questions regarding copper ion regulation of KCTD1 structure and function. For example, can altered copper ion levels associated with protein misfolding diseases affect KCTD1 structure? To address these questions, we performed a set of experiments to determine whether Cu 2+ influences KCTD1 structure, and if so, how this occurs. We report that in the presence of Cu 2+ , KCTD1 tends to form aggregates in a manner that is dependent on the Cu 2+ concentration. We show that KCTD1 aggregation in the presence of Cu 2+ is fibrillar and resistant to proteinase K (PK) digestion. Furthermore, we show that KCTD1 aggregation in the presence of Cu 2+ is functionally cytotoxic. Our studies suggest that Cu 2+ may affect KCTD1 misfolding/ aggregation and lead to cytotoxicity.
doi:10.1038/srep32658 pmid:27596723 pmcid:PMC5011690 fatcat:o4pthmxaofb4dpgy6eqfzl3dnq

Coaxial Electrospun PLLA Fibers Modified with Water-Soluble Materials for Oligodendrocyte Myelination

Zhepeng Liu, Jing Wang, Haini Chen, Guanyu Zhang, Zhuman Lv, Yijun Li, Shoujin Zhao, Wenlin Li
2021 Polymers  
Myelin sheaths are essential in maintaining the integrity of axons. Development of the platform for in vitro myelination would be especially useful for demyelinating disease modeling and drug screening. In this study, a fiber scaffold with a core–shell structure was prepared in one step by the coaxial electrospinning method. A high-molecular-weight polymer poly-L-lactic acid (PLLA) was used as the core, while the shell was a natural polymer material such as hyaluronic acid (HA), sodium alginate
more » ... (SA), or chitosan (CS). The morphology, differential scanning calorimetry (DSC), Fourier transform infrared spectra (FTIR), contact angle, viability assay, and in vitro myelination by oligodendrocytes were characterized. The results showed that such fibers are bead-free and continuous, with an average size from 294 ± 53 to 390 ± 54 nm. The DSC and FTIR curves indicated no changes in the phase state of coaxial brackets. Hyaluronic acid/PLLA coaxial fibers had the minimum contact angle (53.1° ± 0.24°). Myelin sheaths were wrapped around a coaxial electrospun scaffold modified with water-soluble materials after a 14-day incubation. All results suggest that such a scaffold prepared by coaxial electrospinning potentially provides a novel platform for oligodendrocyte myelination.
doi:10.3390/polym13203595 pmid:34685353 fatcat:ifv2cfq5ajb6npn5eo5ujlbdli

Exploration of Quantum Neural Architecture by Mixing Quantum Neuron Designs [article]

Zhepeng Wang, Zhiding Liang, Shanglin Zhou, Caiwen Ding, Yiyu Shi, Weiwen Jiang
2021 arXiv   pre-print
With the constant increase of the number of quantum bits (qubits) in the actual quantum computers, implementing and accelerating the prevalent deep learning on quantum computers are becoming possible. Along with this trend, there emerge quantum neural architectures based on different designs of quantum neurons. A fundamental question in quantum deep learning arises: what is the best quantum neural architecture? Inspired by the design of neural architectures for classical computing which
more » ... y employs multiple types of neurons, this paper makes the very first attempt to mix quantum neuron designs to build quantum neural architectures. We observe that the existing quantum neuron designs may be quite different but complementary, such as neurons from variational quantum circuits (VQC) and Quantumflow. More specifically, VQC can apply real-valued weights but suffer from being extended to multiple layers, while QuantumFlow can build a multi-layer network efficiently, but is limited to use binary weights. To take their respective advantages, we propose to mix them together and figure out a way to connect them seamlessly without additional costly measurement. We further investigate the design principles to mix quantum neurons, which can provide guidance for quantum neural architecture exploration in the future. Experimental results demonstrate that the identified quantum neural architectures with mixed quantum neurons can achieve 90.62% of accuracy on the MNIST dataset, compared with 52.77% and 69.92% on the VQC and QuantumFlow, respectively.
arXiv:2109.03806v2 fatcat:3g77zxuyubdarh6mtpfwd44p6m

Optimizing FPGA-based Accelerator Design for Large-Scale Molecular Similarity Search [article]

Hongwu Peng, Shiyang Chen, Zhepeng Wang, Junhuan Yang, Scott A. Weitze, Tong Geng, Ang Li, Jinbo Bi, Minghu Song, Weiwen Jiang, Hang Liu, Caiwen Ding
2021 arXiv   pre-print
Molecular similarity search has been widely used in drug discovery to identify structurally similar compounds from large molecular databases rapidly. With the increasing size of chemical libraries, there is growing interest in the efficient acceleration of large-scale similarity search. Existing works mainly focus on CPU and GPU to accelerate the computation of the Tanimoto coefficient in measuring the pairwise similarity between different molecular fingerprints. In this paper, we propose and
more » ... timize an FPGA-based accelerator design on exhaustive and approximate search algorithms. On exhaustive search using BitBound & folding, we analyze the similarity cutoff and folding level relationship with search speedup and accuracy, and propose a scalable on-the-fly query engine on FPGAs to reduce the resource utilization and pipeline interval. We achieve a 450 million compounds-per-second processing throughput for a single query engine. On approximate search using hierarchical navigable small world (HNSW), a popular algorithm with high recall and query speed. We propose an FPGA-based graph traversal engine to utilize a high throughput register array based priority queue and fine-grained distance calculation engine to increase the processing capability. Experimental results show that the proposed FPGA-based HNSW implementation has a 103385 query per second (QPS) on the Chembl database with 0.92 recall and achieves a 35x speedup than the existing CPU implementation on average. To the best of our knowledge, our FPGA-based implementation is the first attempt to accelerate molecular similarity search algorithms on FPGA and has the highest performance among existing approaches.
arXiv:2109.06355v1 fatcat:ynsyv7fynrelfdugfgsnrutbo4

Germinal disc region: an appropriate source for obtaining maternal DNA from eggs

Zhepeng Wang, Changliang Yan, Guoqiang Zhang, Guohua Meng, Yu Du, Ruifang Liu
2017 Canadian Journal of Animal Science  
doi:10.1139/cjas-2016-0061 fatcat:fe45gjfmzje4lcudytbpddlil4
« Previous Showing results 1 — 15 out of 65 results