Filters








11,151 Hits in 2.3 sec

Imitation Learning over Heterogeneous Agents with Restraining Bolts

Giuseppe De Giacomo, Marco Favorito, Luca Iocchi, Fabio Patrizi
2020 Zenodo  
A common problem in Reinforcement Learning (RL) is that the reward function is hard to express. This can be overcome by resorting to Inverse Reinforcement Learning (IRL), which consists in first obtaining a reward function from a set of execution traces generated by an expert agent, and then making the learning agent learn the expert's behavior –this is known as Imitation Learning (IL). Typical IRL solutions rely on a numerical representation of the reward function, which raises problems
more » ... ses problems related to the adopted optimization procedures. We describe an IL method where the execution traces generated by the expert agent, possibly via planning, are used to produce a logical (as opposed to numerical) specification of the reward function, to be incorporated into a device known as Restraining Bolt (RB). The RB can be attached to the learning agent to drive the learning process and ultimately make it imitate the expert. We show that IL can be applied to heterogeneous agents, with the expert, the learner and the RB using different representations of the environment's actions and states, without specifying mappings among their representations.
doi:10.5281/zenodo.3944797 fatcat:xqzv4z7oevdr3hzsjrjuqi2ovu

A Standard Grammar for Temporal Logics on Finite Traces [article]

Marco Favorito
2021 arXiv   pre-print
The heterogeneity of tools that support temporal logic formulae poses several challenges in terms of interoperability. In particular, a standard syntax for temporal logic on finite traces, despite similar to the one for infinite traces, is currently missing. This document proposes a standard grammar for several temporal logic formalisms interpreted over finite traces, like Linear Temporal Logic (LTLf), Linear Dynamic Logic (LDLf), Pure-Past Linear Temporal Logic (PLTLf) and Pure-Past Linear
more » ... ure-Past Linear Dynamic Logic (PLDLf).
arXiv:2012.13638v2 fatcat:4vcywtvz5zeghfdbl6wvpxfvmq

Foundations for Restraining Bolts: Reinforcement Learning with LTLf/LDLf Restraining Specifications

Giuseppe De Giacomo, Luca Iocchi, Marco Favorito, Fabio Patrizi
2020 Zenodo  
In this work we investigate on the concept of "restraining bolt", envisioned in Science Fiction. Specifically we introduce a novel problem in AI. We have two distinct sets of features extracted from the world, one by the agent and one by the authority imposing restraining specifications (the "restraining bolt"). The two sets are apparently unrelated since of interest to independent parties, however they both account for (aspects of) the same world. We consider the case in which the agent is a
more » ... ch the agent is a reinforcement learning agent on the first set of features, while the restraining bolt is specified logically using linear time logic on finite traces LTLf/LDLf over the second set of features. We show formally, and illustrate with examples, that, under general circumstances, the agent can learn while shaping its goals to suitably conform (as much as possible) to the restraining bolt specifications.
doi:10.5281/zenodo.3944779 fatcat:jp25lvvx5ndcdi75jmdp3b7vdm

Foundations for Restraining Bolts: Reinforcement Learning with LTLf/LDLf restraining specifications [article]

Giuseppe De Giacomo and Luca Iocchi and Marco Favorito and Fabio Patrizi
2019 arXiv   pre-print
In this work we investigate on the concept of "restraining bolt", envisioned in Science Fiction. Specifically we introduce a novel problem in AI. We have two distinct sets of features extracted from the world, one by the agent and one by the authority imposing restraining specifications (the "restraining bolt"). The two sets are apparently unrelated since of interest to independent parties, however they both account for (aspects of) the same world. We consider the case in which the agent is a
more » ... ch the agent is a reinforcement learning agent on the first set of features, while the restraining bolt is specified logically using linear time logic on finite traces LTLf/LDLf over the second set of features. We show formally, and illustrate with examples, that, under general circumstances, the agent can learn while shaping its goals to suitably conform (as much as possible) to the restraining bolt specifications.
arXiv:1807.06333v2 fatcat:jl2rnvs7zrg25pvubjcpljwiei

Proportional Analysis of Pig Kidney Arterial Segments: Differences from the Human Kidney

Marco Pereira-Sampaio, Luciano Alves Favorito, Robert Henry, Francisco J.B. Sampaio
2007 Journal of endourology  
Purpose: To present a systematic study and a proportional analysis of the arterial segments of the pig kidney. Materials and Methods: Sixty-one three-dimensional endocasts of the arterial segments of pig kidneys were studied. Each segment was injected with a resin of a different color. Cavalieri's principle was used to calculate the volume of each renal segment, and these results were compared with the results from the point-counting planimetry method used on photographs of pig-kidney surfaces.
more » ... ig-kidney surfaces. Results: Two to five renal segments were observed. Division into two segments, a cranial and a caudal, was the most common (42.62%). The renal volume ranged from 101 to 173 cm 3 (mean 130.85 cm 3 ). The cranial segment was present in 39 of the 57 casts (68.42%). It presented the greatest median value of proportional area (50.00%) and also the greatest maximum value of proportional area, accounting for as much as 74.04% of the total kidney area. The ventral segment, which was found in 20 of the 57 casts (35.09%), presented the lowest median value of proportional area (13.87%) and showed the most variation in area (coefficient of variation 72.89%). There was no significant statistical difference between the segmental areas as evaluated by Cavalieri's principle and by the point-counting planimetry method. Conclusions: The distribution and size of the renal-arterial segments in pigs are not similar to those of the human kidneys. Therefore, this information must be taken into account by practitioners of urologic training or ablation using pigs as the animal model, as the structure of the porcine arterial segments cannot be transposed to humans.
doi:10.1089/end.2006.0318 pmid:17705772 fatcat:mgeckvlunvexniz353teka4lum

Temporal Logic Monitoring Rewards via Transducers

Giuseppe De Giacomo, Marco Favorito, Luca Iocchi, Fabio Patrizi, Alessandro Ronca
2020 Proceedings of the Seventeenth International Conference on Principles of Knowledge Representation and Reasoning  
In Markov Decision Processes (MDPs), rewards are assigned according to a function of the last state and action. This is often limiting, when the considered domain is not naturally Markovian, but becomes so after careful engineering of extended state space. The extended states record information from the past that is sufficient to assign rewards by looking just at the last state and action. Non-Markovian Reward Decision Processes (NRMDPs) extend MDPs by allowing for non-Markovian rewards, which
more » ... ian rewards, which depend on the history of states and actions. Non-Markovian rewards can be specified in temporal logics on finite traces such as LTLf/LDLf, with the great advantage of a higher abstraction and succinctness; they can then be automatically compiled into an MDP with an extended state space. We contribute to the techniques to handle temporal rewards and to the solutions to engineer them. We first present an approach to compiling temporal rewards which merges the formula automata into a single transducer, sometimes saving up to an exponential number of states. We then define monitoring rewards, which add a further level of abstraction to temporal rewards by adopting the four-valued conditions of runtime monitoring; we argue that our compilation technique allows for an efficient handling of monitoring rewards. Finally, we discuss application to reinforcement learning.
doi:10.24963/kr.2020/89 dblp:conf/kr/GiacomoFIPR20 fatcat:w3p4x2socbdovjeolfrbmmiy2i

The Dog Kidney as Experimental Model in Endourology: Anatomic Contribution

Marco A. Pereira-Sampaio, Beatriz P.S. Marques-Sampaio, Robert W. Henry, Luciano A. Favorito, Francisco J.B. Sampaio
2009 Journal of endourology  
A systematic study of the morphometry and the collecting system of the canine kidney is presented and compared with previous findings in humans. Renal measurements (kidney length, width, and thickness) were recorded. In addition, 110 three-dimensional endocasts of the kidney collecting system were produced and studied. Anatomic details, important to research and surgical training in endourology, were observed and recorded in canine kidneys. Dogs whose height was more than 70 cm at the withers
more » ... cm at the withers presented similar kidney measurements to those found in the adult human. The collecting system consisted only of a renal pelvis with a variable number of recesses around its perimeter. The dog kidney is not a good model for experimental studies that consider the morphology of the collecting system. Kidneys from dogs taller than 70 cm, however, might be useful as a model in experimental studies in which renal volume is an important aspect, such as shockwave lithotripsy and endourology.
doi:10.1089/end.2008.0552 pmid:19456242 fatcat:dksxcge6unfkzbpvspjlujcn6e

Dog kidney: Anatomical relationships between intrarenal arteries and kidney collecting system

Beatriz P.S. Marques-Sampaio, Marco A. Pereira-Sampaio, Robert W. Henry, Luciano A. Favorito, Francisco J.B. Sampaio
2007 The Anatomical Record  
The detailed findings of canine intrarenal anatomy (collecting system and arteries) are presented. Ninety-five three-dimensional endocasts of the kidney collecting system together with the intrarenal arteries were prepared using standard injection-corrosion techniques and were studied. A single renal artery was observed in 88.4% of the casts. The renal artery divided into a dorsal and a ventral branch. Using the branching pattern of the ventral and dorsal divisions of the renal artery, the
more » ... al artery, the vessels were classified in type I or type II. Type I presented a cranial and a caudal artery, whereas type II presented a mesorenal and a caudal artery. Cranial branches of dorsal and ventral arteries supplied the cranial pole in 90.5% of the specimens. Caudal branches of the dorsal and the ventral divisions of the renal artery irrigated both the caudal pole and the mid-zone of the kidney in 95.8% and 98.9% of the cases, respectively. In all casts, caudal branches of both dorsal and ventral arteries supplied the caudal pole. Therefore, the caudal branches of the ventral and dorsal divisions of the renal artery are of utmost importance in the kidney arterial supply. Although many results of renal and intrarenal anatomy in dogs may not be completely transposed to humans, the anatomical relationship between arteries and the collecting system in the cranial pole of the dog kidney is similar to those in man. This fact supports the use of the dog as an animal model for urologic procedures at the cranial pole. Anat Rec,
doi:10.1002/ar.20567 pmid:17654675 fatcat:hgknpc5tz5e5begglc3k2afnam

Pig Kidney: Anatomical Relationships Between the Renal Venous Arrangement and the Kidney Collecting System

Helio J.S. Bagetti Filho, Marco A. Pereira-Sampaio, Luciano A. Favorito, Francisco J.B. Sampaio
2008 Journal of Urology  
Purpose: We present a systematic study of the anatomical relationship between the intrarenal veins and the kidney collecting system in pigs. Materials and Methods: The intrarenal anatomy (collecting system and veins) was studied in 61, 3-dimensional endocasts of the kidney collecting system together with the intrarenal veins. Results: There are free anastomoses between the intrarenal veins. The interlobar veins unite to produce large venous trunks, which form the renal vein. In our study we
more » ... In our study we observed 2 trunks (cranial and caudal) in 54 of the 61 cases (88.53%) and 3 trunks (cranial, middle and caudal) in 7 (11.47%). Only the ventral surfaces of the cranial and caudal poles were drained by large veins, while the dorsal surfaces emptied by anastomoses into the ventral interlobar veins. There were large veins in a close relationship to the ventral surface (90.16%) and to the dorsal surface (3.28%) of the ureteropelvic junction. In 33 of the 61 cases (54.10%) there was 1 or 2 small dorsal veins. Conclusions: Although some results of intrarenal venous arrangement in pigs could not be completely transposed to humans, many similarities of pig and human kidneys support its use as the best animal model for urological procedures.
doi:10.1016/j.juro.2007.11.040 pmid:18295250 fatcat:zpifemgnonb23hdz6zsqk2usn4

Dry matter productivity and soil physical properties after winter cover crops cultivation

Katiely Aline Anschau Deimling, Edleusa Pereira Seidel, Jean Sérgio Rosset, Marcos Cesar Mottin, Daniela Da Rocha Herrmann, Amanda Cecato Favorito
2020 Scientia Agraria Paranaensis  
The present study aimed to evaluate the dry matter yield of cover crops cultivated in monoculture and intercropped in a no-till system and its effects on the soil physical properties. The experimental design was of randomized blocks, with four replicates. Treatments used were black oat, black oat + forage radish, forage radish, black oat + field pea, field pea and the control (fallow). After 100 days after sowing the cover crops, the dry matter yield was evaluated, with the highest values found
more » ... ighest values found in the intercropped crops. After desiccation, undeformed soil samples were collected for the determination of macroporosity, microporosity, total porosity and soil bulk density in the 0 - 0,10; 0,10 - 0,20; 0,20 - 0,30 and 0,30 - 0,40 m layers. Soil penetration resistance was evaluated with a digital penetrometer. The intercrop of black oat with field pea and with forage radish provided the highest dry matter yield, showing the potential of dry matter accumulation in relation to monoculture. The cover crops were capable of improving the macroporosity, bulk density and soil penetration resistance when compared to the fallow area (control); however, they had no influence in soil aggregation due to the high compaction degree in the area.
doi:10.18188/sap.v18i4.22248 fatcat:aen6zxijzvdexfhxhx3emdzahu

Soil physical properties and soybean productivity in succession to cover crops

Katiely Aline Anschau Deimling, Edleusa Pereira Seidel, Jean Sérgio Rosset, Marcos Cesar Mottin, Daniela da Rocha Herrmann, Amanda Cecato Favorito
2019 Revista Ceres  
The objective of this research was to assess soybean productivity in succession to cover crops grown during the winter, in addition to assessing physical properties macroporosity (Ma), microporosity (Mi), total porosity (Pt), soil density (Sd) and aggregate stability by means of the following variables: aggregate stability index, geometric mean diameter and weighted mean diameter after soybean crop cultivation. The experiment was conducted in the municipality of Quatro Pontes, PR, using a
more » ... , PR, using a randomized-block design with six treatments and four replications. Treatments consisted of soybean crop grown on top of different cover plants' haystack: black oat, black oat + forage turnip, forage turnip, black oat + forage pea, forage pea and control. Significant differences were observed for soil macroporosity and density. At the depth of 0.10 m, the highest Ma was observed in the area with oat and oat + turnip haystack. At other depths, all cover crops were superior to control. Treatments with cover crops were efficient in reducing soil PR. As for soil aggregation, the treatment with pea was superior to control for weighted mean diameter. The treatments with soybean sown after intercropping obtained greater mass, as well as higher productivity.
doi:10.1590/0034-737x201966040010 fatcat:hxsmxowagrcqtjcc57woibotcq

Cranial Pole Nephrectomy in the Pig Model: Anatomic Analysis of Arterial Injuries in Tridimensional Endocasts

Marco A. Pereira-Sampaio, Robert W. Henry, Luciano A. Favorito, Francisco J. B. Sampaio
2012 Journal of endourology  
Purpose: To assess the intrarenal arteries injuries after cranial pole nephrectomy in a pig model to compare these findings with those in humans. Materials and Methods: Polyester resin was injected through the ureter and the renal artery to make threedimensional casts of 61 pig kidneys. The cranial pole of the kidneys was sectioned at four different sites before the solidification of the resin, and the casts were examined for arterial damage. Results: Section performed through the hilus (15
more » ... h the hilus (15 kidneys): The cranial division of the renal artery was sectioned in two (13.33%) cases, the ventral branch of the cranial division of the renal artery was sectioned in 13 (86.7%) cases, and the dorsal branch of the cranial division of the renal artery was sectioned in 11 (73.34%) cases. Section at 0.5 cm cranial to the hilus (16 kidneys): The cranial division of the renal artery was sectioned in 1 (6.25%) case, the ventral branch of the cranial division of the renal artery was sectioned in 14 (87.5%) cases, and the dorsal branch of the cranial division of the renal artery was sectioned in 13 (81.25%) cases. Section at 1.0 cm cranial to the hilus (15 kidneys): The ventral branch of the cranial division of the renal artery was sectioned in five (33.33%) cases, and the dorsal branch of the cranial division of the renal artery was injured in five (33.33%) cases. Section at 1.5 cm cranial to the hilus (15 kidneys): No lesions were found in the main arteries, only in the interlobular branches. Conclusions: As previously demonstrated in humans, sections at 1.0 cm or more cranially to the hilus in pigs also showed a significant decrease in damage to the major intrarenal arteries. Therefore, as regards arterial damage, the pig kidney is a useful model for partial nephrectomy in the cranial (upper) pole. Abbreviations Used LPN ¼ laparoscopic partial nephrectomy OPN ¼ open partial nephrectomy PIG MODEL FOR PARTIAL NEPHRECTOMY 721
doi:10.1089/end.2011.0489 pmid:22204501 fatcat:nbm4nnfryjgq7fnfb5vdxd2na4

PIG KIDNEY: ANATOMICAL RELATIONSHIPS BETWEEN THE INTRARENAL ARTERIES AND THE KIDNEY COLLECTING SYSTEM. APPLIED STUDY FOR UROLOGICAL RESEARCH AND SURGICAL TRAINING

MARCO A. PEREIRA-SAMPAIO, LUCIANO A. FAVORITO, FRANCISCO J.B. SAMPAIO
2004 Journal of Urology  
Purpose: We present a systematic study of the anatomical relationships between intrarenal arteries and the kidney collecting system in pigs. Materials and Methods: The intrarenal anatomy (collecting system and arteries) was studied in 91, 3-dimensional endocasts of the kidney collecting system together with the intrarenal arteries. Results: Some anatomical details that have importance to help research and surgical training in urology when using the pig as an animal model were observed and
more » ... observed and described. It was found that there was only 1 artery per kidney. This artery divided into cranial and caudal branches in 85 cases (93.4%). In 6 cases (6.6%) the primary division of the renal artery was in a dorsal and in a ventral branch. In all cases 2 arteries (1 ventral and 1 dorsal) involved the cranial caliceal group. In the dorsal mid zone a dorsal artery originated from the cranial division of the renal artery, which was obliquely positioned in 47.25% of cases. The arterial supply related to the ventral mid zone of the kidney consisted of branches that coursed horizontally in the ventral surface of the renal pelvis in 81.32% of cases. The caudal division of the renal artery supplied the ventral and dorsal surfaces of the caudal caliceal group in 84.62% of cases, while in 15.38% a dorsal artery supplied its dorsal surface. Conclusions: Although the results of renal and intrarenal anatomy in pigs could not be completely transposed to humans, many similarities in the pig and human intrarenal arteries support its use as the best animal model for urological procedures.
doi:10.1097/01.ju.0000138085.19352.b5 pmid:15540793 fatcat:qr3mfiadynga5e3tylq2ax3hui

Restraining Bolts for Reinforcement Learning Agents

Giuseppe De Giacomo, Luca Iocchi, Marco Favorito, Fabio Patrizi
2020 PROCEEDINGS OF THE THIRTIETH AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE AND THE TWENTY-EIGHTH INNOVATIVE APPLICATIONS OF ARTIFICIAL INTELLIGENCE CONFERENCE  
In this work we have investigated the concept of "restraining bolt", inspired by Science Fiction. We have two distinct sets of features extracted from the world, one by the agent and one by the authority imposing some restraining specifications on the behaviour of the agent (the "restraining bolt"). The two sets of features and, hence the model of the world attainable from them, are apparently unrelated since of interest to independent parties. However they both account for (aspects of) the
more » ... aspects of) the same world. We have considered the case in which the agent is a reinforcement learning agent on a set of low-level (subsymbolic) features, while the restraining bolt is specified logically using linear time logic on finite traces f/f over a set of high-level symbolic features. We show formally, and illustrate with examples, that, under general circumstances, the agent can learn while shaping its goals to suitably conform (as much as possible) to the restraining bolt specifications.1
doi:10.1609/aaai.v34i09.7114 fatcat:hv665kfvjvbhbdwjhmudzfd6ee

Effects of Acute Cadmium Exposure on the Pituitary Gland of Podarcis sicula~!2009-11-18~!2010-03-08~!2010-05-14~!

Rossana Favorito, Maria Consiglio Grimaldi, Marco Coppola, Ida Ferrandino
2010 The Open Zoology Journal  
The Open Zoology Journal, 2010, Volume 3 Favorito et al.  ... 
doi:10.2174/1874336601003020030 fatcat:tlu5ztji4rfgrmjbkrd4b3wel4
« Previous Showing results 1 — 15 out of 11,151 results