Filters








78 Hits in 0.97 sec

Stereo Neural Vernier Caliper [article]

Shichao Li, Zechun Liu, Zhiqiang Shen, Kwang-Ting Cheng
2022 arXiv   pre-print
Estimating depth from a single RGB image is ill-posed and leads to limited 3D object detection performance (Brazil and Liu 2019; Wang et al. 2021a; Lu et al. 2021) .  ... 
arXiv:2203.11018v2 fatcat:gcyejbg4yzda3h76g2naso5p6y

Sliced Recursive Transformer [article]

Zhiqiang Shen and Zechun Liu and Eric Xing
2022 arXiv   pre-print
We present a neat yet effective recursive operation on vision transformers that can improve parameter utilization without involving additional parameters. This is achieved by sharing weights across the depth of transformer networks. The proposed method can obtain a substantial gain (~2%) simply using naive recursive operation, requires no special or sophisticated knowledge for designing principles of networks, and introduces minimal computational overhead to the training procedure. To reduce
more » ... additional computation caused by recursive operation while maintaining the superior accuracy, we propose an approximating method through multiple sliced group self-attentions across recursive layers which can reduce the cost consumption by 10~30% with minimal performance loss. We call our model Sliced Recursive Transformer (SReT), a novel and parameter-efficient vision transformer design that is compatible with a broad range of other designs for efficient ViT architectures. Our best model establishes significant improvement on ImageNet-1K over state-of-the-art methods while containing fewer parameters. The proposed weight sharing mechanism by sliced recursion structure allows us to build a transformer with more than 100 or even 1000 shared layers with ease while keeping a compact size (13~15M), to avoid optimization difficulties when the model is too large. The flexible scalability has shown great potential for scaling up models and constructing extremely deep vision transformers. Code is available at https://github.com/szq0214/SReT.
arXiv:2111.05297v3 fatcat:fz4i3bsowree3jgbtsukr5fppa

Conditional Link Prediction of Category-Implicit Keypoint Detection [article]

Ellen Yi-Ge, Rui Fan, Zechun Liu, Zhiqiang Shen
2020 arXiv   pre-print
Keypoints of objects reflect their concise abstractions, while the corresponding connection links (CL) build the skeleton by detecting the intrinsic relations between keypoints. Existing approaches are typically computationally-intensive, inapplicable for instances belonging to multiple classes, and/or infeasible to simultaneously encode connection information. To address the aforementioned issues, we propose an end-to-end category-implicit Keypoint and Link Prediction Network (KLPNet), which
more » ... the first approach for simultaneous semantic keypoint detection (for multi-class instances) and CL rejuvenation. In our KLPNet, a novel Conditional Link Prediction Graph is proposed for link prediction among keypoints that are contingent on a predefined category. Furthermore, a Cross-stage Keypoint Localization Module (CKLM) is introduced to explore feature aggregation for coarse-to-fine keypoint localization. Comprehensive experiments conducted on three publicly available benchmarks demonstrate that our KLPNet consistently outperforms all other state-of-the-art approaches. Furthermore, the experimental results of CL prediction also show the effectiveness of our KLPNet with respect to occlusion problems.
arXiv:2011.14462v1 fatcat:etbfwctuifa4leqdxrvbcoehzu

BiT: Robustly Binarized Multi-distilled Transformer [article]

Zechun Liu, Barlas Oguz, Aasish Pappu, Lin Xiao, Scott Yih, Meng Li, Raghuraman Krishnamoorthi, Yashar Mehdad
2022 arXiv   pre-print
Surprisingly, researchers in computer vision have been able to demonstrate BNNs with remarkable accuracy (Liu et al., 2018; Qin et al., 2020; Martinez et al., 2020) .  ...  2016) , by expanding Eq. 3, we have J (α) = α 2 XB T XB − 2αX R T XB + X R T X R (4) For the layers with X R ∈ R n we follow the traditional methods of binarizing activations (Rastegari et al., 2016; Liu  ...  Bohan Zhuang, Chunhua Shen, Mingkui Tan, Lingqiao Liu, and Ian Reid. Towards effective lowbitwidth convolutional neural networks.  ... 
arXiv:2205.13016v1 fatcat:devve35hfvblxf7w3bxuzssdx4

How Do Adam and Training Strategies Help BNNs Optimization? [article]

Zechun Liu, Zhiqiang Shen, Shichao Li, Koen Helwegen, Dong Huang, Kwang-Ting Cheng
2021 arXiv   pre-print
Correspondence to: Zhiqiang Shen <zhiqians@andrew.cmu.edu>, Zechun Liu <zechun.liu@connect.ust.hk>. Proceedings of the 38 th International Conference on Machine Learning, PMLR 139, 2021.  ...  Networks BOPs ×10 9 FLOPs ×10 8 OPs ×10 8 XNOR-ResNet-18 (Rastegari et al., 2016) 1.70 1.41 1.67 Bi-RealNet-18 (Liu et al., 2018b) 1.68 1.39 1.63 ReActNet-A (Liu et al., 2020) 4.82 0.12 0.87  ... 
arXiv:2106.11309v1 fatcat:5m3ezp4gxvdsbdg2ptv2udpzmq

Binarizing MobileNet via Evolution-based Searching [article]

Hai Phan, Zechun Liu, Dang Huynh, Marios Savvides, Kwang-Ting Cheng, Zhiqiang Shen
2020 arXiv   pre-print
Liu et al. [28] proposed DARTS, a prominent gradient-based method that optimizes jointly one-shot models on a continuous relaxation of the search space.  ... 
arXiv:2005.06305v2 fatcat:wqpnfyedt5ey7fztndu5atdhfq

Automatic Generation Control Considering Uncertainties of the Key Parameters in the Frequency Response Model [article]

Likai Liu, Zechun Hu, Asad Mujeeb
2021 arXiv   pre-print
The highly fluctuated renewable generations and electric vehicles have undergone tremendous growth in recent years. The majority of them are connected to the grid via power electronic devices, resulting in wide variation ranges for several key parameters in the frequency response model (FRM) such as system inertia and load damping factor. In this paper, an automatic generation control (AGC) method considering the uncertainties of these key parameters is proposed. First, the historical power
more » ... em operation data following large power disturbances are used to identify the FRM key parameters offline. Second, the offline identification results and the normal operation data prior to the occurrence of the disturbance are used to train the online probability estimation model of the FRM key parameters. Third, the online estimation results of the FRM key parameters are used as the input, and the model predictive-based AGC signal optimization method is developed based on distributionally robust optimization (DRO) technology. Case studies conducted on the IEEE 118-Bus System show that the proposed AGC method outperforms the widely utilized PI-based control method in terms of performance and efficiency.
arXiv:2108.05672v1 fatcat:ky7kuftr4rfkzdgflbiuwexfku

Modeling of Frequency Security Constraints and Quantification of Frequency Control Reserve Requirements for Unit Commitment [article]

Likai Liu, Zechun Hu
2021 arXiv   pre-print
The high penetration of converter-based renewable energy sources has brought challenges to the power system frequency control. It is essential to consider the frequency security constraints and frequency control reserve requirements in unit commitment (UC). Considering that the risk of frequency insecurity varies under the changeable operational condition, we propose to optimize the PFC droop gains and reserve capacities in the UC model to provide diverse control efforts in different risk
more » ... adaptively. Copula theory is used to establish the joint distribution model among frequency control performance, secondary frequency control (SFC) reserve capacities, and power fluctuations. Then the distributionally robust optimization technique is utilized in the SFC reserve requirement determination to handle the possible error in the probability model. The UC simulation is conducted on IEEE 118-bus system to test the proposed optimal PFC droop gain strategy and SFC reserve requirement quantification method. Simulation results show that the proposed optimal PFC droop gain strategy is better than the traditional fixed PFC droop gain setting on economic efficiency and operational flexibility. Besides, the SFC reserve requirement calculated by the proposed method is more appropriate than the actual SFC reserve capacity in the historical operation.
arXiv:2110.13448v1 fatcat:hk7lmcdjofdzfobwverjerd2iy

Bi-Real Net: Binarizing Deep Network Towards Real-Network Performance [article]

Zechun Liu, Wenhan Luo, Baoyuan Wu, Xin Yang, Wei Liu, Kwang-Ting Cheng
2019 arXiv   pre-print
In this paper, we study 1-bit convolutional neural networks (CNNs), of which both the weights and activations are binary. While efficient, the lacking of representational capability and the training difficulty impede 1-bit CNNs from performing as well as real-valued networks. We propose Bi-Real net with a novel training algorithm to tackle these two challenges. To enhance the representational capability, we propagate the real-valued activations generated by each 1-bit convolution via a
more » ... -free shortcut. To address the training difficulty, we propose a training algorithm using a tighter approximation to the derivative of the sign function, a magnitude-aware gradient for weight updating, a better initialization method, and a two-step scheme for training a deep network. Experiments on ImageNet show that an 18-layer Bi-Real net with the proposed training algorithm achieves 56.4% top-1 classification accuracy, which is 10% higher than the state-of-the-arts (e.g., XNOR-Net) with greater memory saving and lower computational cost. Bi-Real net is also the first to scale up 1-bit CNNs to an ultra-deep network with 152 layers, and achieves 64.5% top-1 accuracy on ImageNet. A 50-layer Bi-Real net shows comparable performance to a real-valued network on the depth estimation task with only a 0.3% accuracy gap.
arXiv:1811.01335v2 fatcat:qxqcwipzmnhmjhkkqys4mci6si

Dibutyl 5-[(4-ethoxycarbonylphenyl)diazenyl]benzene-1,3-dicarboxylate

Ying Liu, Xianxi Zhang, Zechun Xue, Jian Sheng
2010 Acta Crystallographica Section E  
Key indicators: single-crystal X-ray study; T = 293 K; mean (C-C) = 0.006 Å; R factor = 0.064; wR factor = 0.218; data-to-parameter ratio = 14.2. organic compounds o1730 Liu et al.  ... 
doi:10.1107/s1600536810022762 pmid:21587947 pmcid:PMC3007049 fatcat:aquzli57bnhphelis5gzaup7w4

Vision Transformer Slimming: Multi-Dimension Searching in Continuous Optimization Space [article]

Arnav Chavan and Zhiqiang Shen and Zhuang Liu and Zechun Liu and Kwang-Ting Cheng and Eric Xing
2022 arXiv   pre-print
This paper explores the feasibility of finding an optimal sub-model from a vision transformer and introduces a pure vision transformer slimming (ViT-Slim) framework. It can search a sub-structure from the original model end-to-end across multiple dimensions, including the input tokens, MHSA and MLP modules with state-of-the-art performance. Our method is based on a learnable and unified ℓ_1 sparsity constraint with pre-defined factors to reflect the global importance in the continuous searching
more » ... space of different dimensions. The searching process is highly efficient through a single-shot training scheme. For instance, on DeiT-S, ViT-Slim only takes 43 GPU hours for the searching process, and the searched structure is flexible with diverse dimensionalities in different modules. Then, a budget threshold is employed according to the requirements of accuracy-FLOPs trade-off on running devices, and a re-training process is performed to obtain the final model. The extensive experiments show that our ViT-Slim can compress up to 40 FLOPs on various vision transformers while increasing the accuracy by 0.6 ImageNet. We also demonstrate the advantage of our searched models on several downstream datasets. Our code is available at https://github.com/Arnav0400/ViT-Slim.
arXiv:2201.00814v2 fatcat:2ia77hhmfndb7blaqry4sdjejm

Partial Is Better Than All: Revisiting Fine-tuning Strategy for Few-shot Learning [article]

Zhiqiang Shen and Zechun Liu and Jie Qin and Marios Savvides and Kwang-Ting Cheng
2021 arXiv   pre-print
Real et al. 2017; Xie and Yuille 2017; Liu et al. 2017; Real et al. 2019) .  ... 
arXiv:2102.03983v1 fatcat:u37noahnpze4jm6ba4pcqihdyu

1-Phenyl-3-(2,4,6-trimethoxyphenyl)prop-2-en-1-one

Ying Liu, Xianxi Zhang, Zechun Xue, Chunyan Lv
2009 Acta Crystallographica Section E  
E65, o2724 [https://doi.org/10.1107/S1600536809040227]1-Phenyl-3-(2,4,6-trimethoxyphenyl)prop-2-en-1-one Ying Liu, Xianxi Zhang, Zechun Xue and Chunyan Lv S1. Comment ).  ...  TINRIS -Oxido-bis({4,4 0 -dibromo-2,2 0 -[ethane-1,2-diylbis(nitrilomethylidyne)]diphenolato}manganese(III)) Liu, Dou, Niu & Zhang (2007a) 10.1107/S1600536807051008 GIMZAE Bis[N-(8-quinolyl)pyridine-  ... 
doi:10.1107/s1600536809040227 pmid:21578321 pmcid:PMC2971127 fatcat:t22miy5tg5d7rjqtwzgdopgdsq

catena-Poly[[aqua(2,2′-bipyridyl)cobalt(II)]-μ-5-nitroisophthalato]

Ying Liu, Qingpeng He, Xianxi Zhang, Zechun Xue, Chunyan Lv
2008 Acta Crystallographica Section E  
2,2 0 -bipyridine-2 N,N 0 )nickel(II)]--oxalato-4 O 1 ,O 2 :O 1 0 ,O 2 0 ] Li, Yan et al. (2008) 10.1107/S1600536808028389 NOHYUF catena-Poly[[aqua(2,2 0 -bipyridyl)cobalt(II)]--5-nitroisophthlalato] Liu  ... 
doi:10.1107/s1600536808038178 pmid:21581202 pmcid:PMC2960127 fatcat:ekyaoyee35bonor572jxb2aqza

The Effects of Cigarette Filter Ventilation on Delivery and Retention of Organic Acids

Deng Qixin, Xie Wei, Liu Zechun, Liu Jiangsheng, Zhang Tingui, Lian Fenyan, Chen Kunyan, Xie Jianping, Liu Huimin, Nie Cong, Yan Quanping
2021 Contributions to Tobacco & Nicotine Research  
The Effects of Cigarette Filter Ventilation on Delivery and Retention of Organic Acids * by Deng Qixin 1 , Xie Wei 1 , Liu Zechun 1 , Liu Jiangsheng 1 , Zhang Tingui 1 , Lian Fenyan 1 , Chen Kunyan 1 ,  ...  Xie Jianping 2 , Liu Huimin 2 , Nie Cong 2 , and Yan Quanping 2 1 Technical Centre of Fujian Tobacco Industrial Corporation, Xiamen, Fujian, P.R.  ... 
doi:10.2478/cttr-2021-0015 fatcat:xgac4kmjp5fkbeuw6bshju6eaq
« Previous Showing results 1 — 15 out of 78 results