Ibrahim El Shar
About
I am machine learning research scientist at Hitachi America in California. I work on reinforcement learning and more broadly sequential decision-making from the perspective of both operations research and machine learning. Recenltly, I've also been working on Generative AI and Large Language Models. I conduct innovative research in optimization and machine learning to design novel machine learning methods and solve industrial challenges. I received my Ph.D. in Operations Research from the University of Pittsburgh and my master's in Operations Research and Financial Engineering from the American University of Beirut.
News
- Decemeber 2023: I will be at NeurIPS 2023 (New Orleans, Dec 10th - Dec 16th) to present our paper, “Weakly Coupled Deep Q-Networks”. Join our poster session on Tuesday, Dec 12, 5:15 p.m. CST, Great Hall & Hall B1+B2 (level 1) Poster #216.
- November 2023: Excited to serve as a panelist for the panel "Navigating Today’s Supply Chain" in the Supply Chain Expo that is organized by the University of Louisville and Western Kentucky University, on Nov 9, 2023. In the panel, I discussed the significance of reinforcement learning in optimizing the supply chain and highlighted the great potential of generative AI across the supply chain, while also addressing its current limitations.
- October 2023: I will be presenting my work on weakly coupled MDPs at INFORMS 2023 Annual meeting at Phoenix, AZ.
- September 2023: Our paper on "Weakly Coupled Deep Q-Networks" got accepted to NeurIPS 2023!
- August 2023: New paper on "Multi-Objective Reinforcement Learning for Sustainable Supply Chain Optimization" got accepted to IEEE International Conference on Automation Science and Engineering (CASE 2023)
- August 2023: I will be attending KDD 2023. Hit me up if you will be there too!
- August 2022: Our paper on "Deep Reinforcement Learning toward Robust Multi-echelon Supply Chain Inventory Optimization" got accepted to IEEE International Conference on Automation Science and Engineering (CASE 2022)
- March 2022: Submitted a new paper titled "Deep Reinforcement Learning toward Robust Multi-echelon Supply Chain Inventory Optimization" to IEEE International Conference on Automation Science and Engineering (CASE). This paper is based on my work during my internship at Hitachi America, Ltd. R&D.
- February 2022: Excited to join Amazon as a Research Scientist Intern!
- June 2021: Will start work as a research machine learning intern for Hitachi America LTD industrial AI lab!
- November 11, 2020: Will present my work at Virtual 2020 INFORMS Annual Meeting.
- July 14, 2020: Join our poster session at ICML 2020 to discuss LBQL!
- June 1, 2020: Our paper “Lookahead-bounded Q-learning” got accepted to ICML2020.
- October 20 - 23, 2019: At INFORMS Annual meeting 2019 to give a talk about Lookahead-Bounded QL.
Publications
Weakly Coupled Deep Q-Networks
Ibrahim El Shar, Daniel R. Jiang.
Advances in neural information processing systems 37 (NeurIPS 2023).
We propose Weakly Coupled Deep Q-Networks (WCDQN) a new deep RL algorithm for weakly coupled MDPs.
Lookahead-Bounded Q-Learning
Ibrahim El Shar, Daniel R. Jiang.
The International Conference on Machine Learning (ICML 2020).
We propose a new provably convergent variant of Q-learning that leverages upper and lower bounds derived using information relaxation techniques.
Multi-Objective Reinforcement Learning for Sustainable Supply Chain Optimization
Ibrahim El Shar, Haiyan Wang, Chetan Gupta.
IEEE International Conference on Automation Science and Engineering (CASE) 2023.
Deep Reinforcement Learning toward Robust Multi-echelon Supply Chain Inventory Optimization
Ibrahim El Shar, Wenhuan Sun, Haiyan Wang, Chetan Gupta.
IEEE International Conference on Automation Science and Engineering (CASE) 2022.
This paper is based on my work during my internship at Hitachi America, Ltd. R&D.
Spatial Dynamic Pricing for Shared-Resource Systems
Ibrahim El Shar, Daniel Jiang.
Work in Progress.
We study the problem of spatial dynamic Pricing for a fixed number of shared resources that circulate in a network. For the general network, we show that the optimal value function is concave and for a network composed of two locations, we show that the optimal policy enjoys certain monotonicity and bounded sensitivity properties. We use these results to propose a novel heuristic and a deep reinforcement learning (DRL) algorithm.
Continuous inventory control with stochastic and non-stationary Markovian demand
Walid W. Nasr, Ibrahim El Shar.
European Journal of Operational Research, 2018.
Projects
We work in an endless grid-world with the task of collecting valuable items and tools. Collecting rewards efficiently involves taking the shortest path to collect items and having a high level plan that guides the agent and prioritize where to go and what items to collect. We propose a novel greedy heuristic that we compare to Deep Q-networks (DQN), imitation learning and Deep Q-learning from Demonstrations (DQfD).
We propose a novel sentence-based text representation for language classification problems, which we evaluate using the IMDB Movie Review Dataset.
We study the problem of dynamic airline pricing when the customers can return their tickets for a posted price each period. The goal is to optimize the prices to set for flight tickets pt and the value of returned tickets prt at the beginning of each period t.
We study the case: A Prescription for Budget Woes at Gracious University Hospital. We develop a simulation model using Python 3.6 and SymPy 1.4 to come up with an automated dispensing cabinets (s, S) restocking policy that satisfies both patients and hospital personnel requirements.
Other work, interests, and hobbies
- Art: "I am seeking, I am stiving, I am in it with all my heart", Vincent Van Gogh. I am a self-taught artist that is driven by passion, seized by obsession and delighted by creation. Art has always been a huge passion of mine. I like to draw portraits and paint landscapes. Check out some of my work here.
- Proud to be a PITT Panther! I exercise daily and love going on walks and hikes.
- I am a big fan of chess, soccer and music.