2,044 Hits in 1.1 sec

Designing Rewards for Fast Learning [article]

Henry Sowerby, Zhiyuan Zhou, Michael L. Littman
2022 arXiv   pre-print
To convey desired behavior to a Reinforcement Learning (RL) agent, a designer must choose a reward function for the environment, arguably the most important knob designers have in interacting with RL agents. Although many reward functions induce the same optimal behavior (Ng et al., 1999), in practice, some of them result in faster learning than others. In this paper, we look at how reward-design choices impact learning speed and seek to identify principles of good reward design that quickly
more » ... uce target behavior. This reward-identification problem is framed as an optimization problem: Firstly, we advocate choosing state-based rewards that maximize the action gap, making optimal actions easy to distinguish from suboptimal ones. Secondly, we propose minimizing a measure of the horizon, something we call the "subjective discount", over which rewards need to be optimized to encourage agents to make optimal decisions with less lookahead. To solve this optimization problem, we propose a linear-programming based algorithm that efficiently finds a reward function that maximizes action gap and minimizes subjective discount. We test the rewards generated with the algorithm in tabular environments with Q-Learning, and empirically show they lead to faster learning. Although we only focus on Q-Learning because it is perhaps the simplest and most well understood RL algorithm, preliminary results with R-max (Brafman and Tennenholtz, 2000) suggest our results are much more general. Our experiments support three principles of reward design: 1) consistent with existing results, penalizing each step taken induces faster learning than rewarding the goal. 2) When rewarding subgoals along the target trajectory, rewards should gradually increase as the goal gets closer. 3) Dense reward that's nonzero on every state is only good if designed carefully.
arXiv:2205.15400v1 fatcat:rqmvbxtetveknpzl4iw76cq24u

Generation of Optical Vortices with Polarization-insensitive Metasurfaces

Zhe Shen, Rui Li, Yingze Xue, Zhiyuan Qiu, Zhiyuan Xiang, Jiayi Zhou, Baifu Zhang
2020 IEEE Photonics Journal  
Traditionally, optical vortices (OVs) were generated with diffractive optical elements (DOEs) such as spiral phase plate (SPP), fork grating, spatial light modulator (SLM), and liquid crystal display (LCD). Here, a method was proposed for generating OVs by employing all-dielectric polarization-insensitive metasurfaces with cylinder arrays, which have high transmission efficiency. The polarization insensitivity of the metasurfaces was illustrated with the incidence of two pairs of orthogonal
more » ... rization, both the phase and transmission efficiency were consistent for the cylinder unit cell, and similar OVs were generated with the cylinder array. The topological charges of the generated OVs can be adjusted through the design of the metasurfaces. OVs with additional characteristics as vector beams, focused beams and Bessel beams were further generated. This work has potential applications in beam shaping, optical tweezers, and optical communication.
doi:10.1109/jphot.2020.3014918 fatcat:fqiibvyju5e57oajrcifhcsuum

Hamartoma of mature cardiomyocytes in right atrium

Xinqi Zhou, Yangzhao Zhou, Yuan Zhaoshun, Mu Zeng, Xinmin Zhou, Xiaobo Liao, Zhiyuan Zhang
2019 Medicine  
Tumors in the heart are rare. Myxomas, rhabdomyomas, and fibromas are the most common benign cardiac tumors. Hamartoma of mature cardiomyocytes (HMCM) is another benign cardiac tumor, are very rare and have only been reported in a few literatures. We report a case of 41-year-old male who suffered short of breath for 3 years, and lower limbs edema for 2 years. Transthoracic echocardiogram (TTE) and cardiac magnetic resonance (CMR) showed a large amount of pericardial effusion and confirmed a
more » ... of 18 × 14 mm on the superior vena cava near the outer edge of right atrium. The patient was first diagnosed as pleural mesothelioma. Surgery was performed to relieve the symptoms and confirm diagnoses. However, during surgery, we found the right atrium is apparently thicken with rough and uneven surface. Histology of right atrium mass indicated it as hamartoma of mature cardiomyocytes. We resected the thicken atrial wall completely, reconstructed right atrium with bovine pericardial patch, and resected the pericardium. Patient was discharged 9 days after surgery, and remained asymptomatic during 9 months follow up. Hamartoma of mature cardomyocytes is a rare benign cardiac tumor. There were 26 cases reported until now. The conclusive diagnosis depends on pathological sections. For patients with symptoms, surgery is an effective treatment for HMCM.
doi:10.1097/md.0000000000016640 pmid:31374034 pmcid:PMC6709070 fatcat:rx5b2go5bbeepaza2daeardodi

The Power of Triply Complementary Priors for Image Compressive Sensing [article]

Zhiyuan Zha, Xin Yuan, Joey Tianyi Zhou, Jiantao Zhou, Bihan Wen, Ce Zhu
2020 arXiv   pre-print
Recent works that utilized deep models have achieved superior results in various image restoration applications. Such approach is typically supervised which requires a corpus of training images with distribution similar to the images to be recovered. On the other hand, the shallow methods which are usually unsupervised remain promising performance in many inverse problems, , image compressive sensing (CS), as they can effectively leverage non-local self-similarity priors of natural images.
more » ... er, most of such methods are patch-based leading to the restored images with various ringing artifacts due to naive patch aggregation. Using either approach alone usually limits performance and generalizability in image restoration tasks. In this paper, we propose a joint low-rank and deep (LRD) image model, which contains a pair of triply complementary priors, namely external and internal, deep and shallow, and local and non-local priors. We then propose a novel hybrid plug-and-play (H-PnP) framework based on the LRD model for image CS. To make the optimization tractable, a simple yet effective algorithm is proposed to solve the proposed H-PnP based image CS problem. Extensive experimental results demonstrate that the proposed H-PnP algorithm significantly outperforms the state-of-the-art techniques for image CS recovery such as SCSNet and WNNM.
arXiv:2005.07902v1 fatcat:2qw47bvo7bacheficmdg2on5yu

Efficient Human Pose Estimation by Maximizing Fusion and High-Level Spatial Attention [article]

Zhiyuan Ren, Yaohai Zhou, Yizhe Chen, Ruisong Zhou, Yayu Gao
2021 arXiv   pre-print
In this paper, we propose an efficient human pose estimation network -- SFM (slender fusion model) by fusing multi-level features and adding lightweight attention blocks -- HSA (High-Level Spatial Attention). Many existing methods on efficient network have already taken feature fusion into consideration, which largely boosts the performance. However, its performance is far inferior to large network such as ResNet and HRNet due to its limited fusion operation in the network. Specifically, we
more » ... nd the number of fusion operation by building bridges between two pyramid frameworks without adding layers. Meanwhile, to capture long-range dependency, we propose a lightweight attention block -- HSA, which computes second-order attention map. In summary, SFM maximizes the number of feature fusion in a limited number of layers. HSA learns high precise spatial information by computing the attention of spatial attention map. With the help of SFM and HSA, our network is able to generate multi-level feature and extract precise global spatial information with little computing resource. Thus, our method achieve comparable or even better accuracy with less parameters and computational cost. Our SFM achieve 89.0 in PCKh@0.5, 42.0 in PCKh@0.1 on MPII validation set and 71.7 in AP, 90.7 in AP@0.5 on COCO validation with only 1.7G FLOPs and 1.5M parameters. The source code will be public soon.
arXiv:2107.13693v1 fatcat:k5l5pipz7bgazgycxy6w5uvxqi

Category Enhanced Word Embedding [article]

Chunting Zhou, Chonglin Sun, Zhiyuan Liu, Francis C.M. Lau
2015 arXiv   pre-print
Distributed word representations have been demonstrated to be effective in capturing semantic and syntactic regularities. Unsupervised representation learning from large unlabeled corpora can learn similar representations for those words that present similar co-occurrence statistics. Besides local occurrence statistics, global topical information is also important knowledge that may help discriminate a word from another. In this paper, we incorporate category information of documents in the
more » ... ning of word representations and to learn the proposed models in a document-wise manner. Our models outperform several state-of-the-art models in word analogy and word similarity tasks. Moreover, we evaluate the learned word vectors on sentiment analysis and text classification tasks, which shows the superiority of our learned word vectors. We also learn high-quality category embeddings that reflect topical meanings.
arXiv:1511.08629v2 fatcat:vnywzjcsfvgmlfcdsrgwetp26m

Evaluating Modules in Graph Contrastive Learning [article]

Ganqu Cui, Yufeng Du, Cheng Yang, Jie Zhou, Liang Xu, Xing Zhou, Xingyi Cheng, Zhiyuan Liu
2022 arXiv   pre-print
The recent emergence of contrastive learning approaches facilitates the application on graph representation learning (GRL), introducing graph contrastive learning (GCL) into the literature. These methods contrast semantically similar and dissimilar sample pairs to encode the semantics into node or graph embeddings. However, most existing works only performed model-level evaluation, and did not explore the combination space of modules for more comprehensive and systematic studies. For effective
more » ... odule-level evaluation, we propose a framework that decomposes GCL models into four modules: (1) a sampler to generate anchor, positive and negative data samples (nodes or graphs); (2) an encoder and a readout function to get sample embeddings; (3) a discriminator to score each sample pair (anchor-positive and anchor-negative); and (4) an estimator to define the loss function. Based on this framework, we conduct controlled experiments over a wide range of architectural designs and hyperparameter settings on node and graph classification tasks. Specifically, we manage to quantify the impact of a single module, investigate the interaction between modules, and compare the overall performance with current model architectures. Our key findings include a set of module-level guidelines for GCL, e.g., simple samplers from LINE and DeepWalk are strong and robust; an MLP encoder associated with Sum readout could achieve competitive performance on graph classification. Finally, we release our implementations and results as OpenGCL, a modularized toolkit that allows convenient reproduction, standard model and module evaluation, and easy extension. OpenGCL is available at .
arXiv:2106.08171v2 fatcat:t3ruixdbazepndw2jk4cyl3yde

Status from a Random Field: How Densely Should One Update? [article]

Zhiyuan Jiang, Sheng Zhou
2019 arXiv   pre-print
In many applications, status information of a general spatial process, in contrast to a point information source, is of interest. In this paper, we consider a system where status information is drawn from a random field and transmitted to a fusion center through a wireless multiaccess channel. The optimal density of spatial sampling points to minimize the remote status estimation error is investigated. Assuming a one-dimensional Gauss Markov random field and an exponential correlation function,
more » ... closed-form expressions of remote estimation error are obtained for First-Come First-Served (FCFS) and Last-Come First-Served (LCFS) service disciplines. The optimal spatial sampling density for the LCFS case is given explicitly. Simulation results are presented which agree with our analysis.
arXiv:1901.05096v1 fatcat:65epstm5hndvpbpssr4yloq6fq

Fourier-Like Transforms of Stable Graphs and Holomorphic Anomaly Equations [article]

Zhiyuan Wang, Jian Zhou
2019 arXiv   pre-print
In this paper we develop a theory of Fourier-like transforms on the space of stable graphs. In particular, we introduce a duality theory of stable graphs. As an application, we derive the holomorphic anomaly equations for general propagators in the work of Aganagic, Bouchard and Klemm.
arXiv:1905.03436v1 fatcat:dcxuzrbanzfp3i2vx6itlzbj4i

Orbifold Euler Characteristics of ℳ_g,n [article]

Zhiyuan Wang, Jian Zhou
2021 arXiv   pre-print
We solve the problem of the computation of the orbifold Euler characteristics of _g,n. We take the works of Harer-Zagier and Bini-Harer as our starting point, and apply the formalisms developed in and to this problem. These formalisms are typical examples of mathematical methods inspired by quantum field theories. We also present many closed formulas and some numerical data. In genus zero the results are related to Ramanujan polynomials, and in higher genera we get recursion relations almost
more » ... ntical to the recursion relations for Ramanujan polynomials but with different initial values. We also show that the generating series given by the orbifold Euler characteristics of ℳ_g,n is the logarithm of the KP tau-function of the topological 1D gravity evaluated at the times given by the orbifold Euler characteristics of ℳ_g,n. Conversely, the logarithm of this tau-function evaluated at the times given by certain generating series of the orbifold Euler characteristics of ℳ_g,n is a generating series of the orbifold Euler characteristics of ℳ_g,n. This is a new example of open-closed duality.
arXiv:1812.10638v2 fatcat:zk77f7bcovhkpd6xb4dgeicosq

NumNet: Machine Reading Comprehension with Numerical Reasoning [article]

Qiu Ran, Yankai Lin, Peng Li, Jie Zhou, Zhiyuan Liu
2019 arXiv   pre-print
Numerical reasoning, such as addition, subtraction, sorting and counting is a critical skill in human's reading comprehension, which has not been well considered in existing machine reading comprehension (MRC) systems. To address this issue, we propose a numerical MRC model named as NumNet, which utilizes a numerically-aware graph neural network to consider the comparing information and performs numerical reasoning over numbers in the question and passage. Our system achieves an EM-score of
more » ... 6% on the DROP dataset, outperforming all existing machine reading comprehension models by considering the numerical relations among numbers.
arXiv:1910.06701v1 fatcat:p5ntlou32vc7hhdtz2zrz2d25y

Token-Based Access Control

Guohua Gan, E Chen, Zhiyuan Zhou, Yan Zhu
2020 IEEE Access  
ZHIYUAN ZHOU received the B.E. degree from the School of Computer and Communication Engineering, University of Science and Technology Beijing, where he is currently a Graduate Student.  ... 
doi:10.1109/access.2020.2979746 fatcat:i5pmfc3dpbbqljnyqn4zhhfzci

Metasurface enabled quantum edge detection

Junxiao Zhou, Shikai Liu, Haoliang Qian, Yinhai Li, Hailu Luo, Shuangchun Wen, Zhiyuan Zhou, Guangcan Guo, Baosen Shi, Zhaowei Liu
2020 Science Advances  
Photo credit: Junxiao Zhou, University of California, San Diego. Fig. 3 . 3 Characterizations of the entangled source.  ...  Photo credit: Junxiao Zhou, University of California, San Diego. Fig. 5 . 5 Entanglement-enabled quantum edge detection has high SNR.  ... 
doi:10.1126/sciadv.abc4385 pmid:33328227 pmcid:PMC7744082 fatcat:6446ebyisjanpijfowpaakwqxm

Magnon-mediated interlayer coupling in an all-antiferromagnetic junction [article]

Yongjian Zhou, Liyang Liao, Xiaofeng Zhou, Hua Bai, Mingkun Zhao, Caihua Wan, Siqi Yin, Lin Huang, Tingwen Guo, Lei Han, Ruyi Chen, Zhiyuan Zhou (+3 others)
2021 arXiv   pre-print
The interlayer coupling mediated by fermions in ferromagnets brings about parallel and anti-parallel magnetization orientations of two magnetic layers, resulting in the giant magnetoresistance, which forms the foundation in spintronics and accelerates the development of information technology. However, the interlayer coupling mediated by another kind of quasi-particle, boson, is still lacking. Here we demonstrate such a static interlayer coupling at room temperature in an antiferromagnetic
more » ... ion Fe2O3/Cr2O3/Fe2O3, where the two antiferromagnetic Fe2O3 layers are functional materials and the antiferromagnetic Cr2O3 layer serves as a spacer. The N\'eel vectors in the top and bottom Fe2O3 are strongly orthogonally coupled, which is bridged by a typical bosonic excitation (magnon) in the Cr2O3 spacer. Such an orthogonally coupling exceeds the category of traditional collinear interlayer coupling via fermions in ground state, reflecting the fluctuating nature of the magnons, as supported by our magnon quantum well model. Besides the fundamental significance on the quasi-particle-mediated interaction, the strong coupling in an antiferromagnetic magnon junction makes it a realistic candidate for practical antiferromagnetic spintronics and magnonics with ultrahigh-density integration.
arXiv:2101.08665v1 fatcat:3uhxghggufbujhkqggi6w23uje

Characterizing the Action-Generalization Gap in Deep Q-Learning [article]

Zhiyuan Zhou, Cameron Allen, Kavosh Asadi, George Konidaris
2022 arXiv   pre-print
We study the action generalization ability of deep Q-learning in discrete action spaces. Generalization is crucial for efficient reinforcement learning (RL) because it allows agents to use knowledge learned from past experiences on new tasks. But while function approximation provides deep RL agents with a natural way to generalize over state inputs, the same generalization mechanism does not apply to discrete action outputs. And yet, surprisingly, our experiments indicate that Deep Q-Networks
more » ... QN), which use exactly this type of function approximator, are still able to achieve modest action generalization. Our main contribution is twofold: first, we propose a method of evaluating action generalization using expert knowledge of action similarity, and empirically confirm that action generalization leads to faster learning; second, we characterize the action-generalization gap (the difference in learning performance between DQN and the expert) in different domains. We find that DQN can indeed generalize over actions in several simple domains, but that its ability to do so decreases as the action space grows larger.
arXiv:2205.05588v1 fatcat:lpzxoeobbzhdxc2k6yw42gquka
« Previous Showing results 1 — 15 out of 2,044 results