With a background in mathematics and operations research, Andreea’s interests lie in complex systems, predominantly in business and economics. She’s passionate about bringing math, data, and creativity to tangible decision problems.
Ciprian Borodescu: I’m here with Andreea Georgescu, Operations Researcher at MIT. I’m super excited, and it’s an honor to have you on this podcast. Thank you so much for being here.
Andreea Georgescu: Thank you. It’s a pleasure to be part of this podcast, especially in the early days. I’m very excited to have a chat with you.
Ciprian Borodescu: We met a few months ago through a common acquaintance, and I was super impressed with the work you’re doing at MIT. Could you share a bit about yourself and the field of research you’re in?
Andreea Georgescu: Yeah, certainly. So I’m originally from Romania, but I moved to the States for college. I’m a mathematician at heart and by training. So, my college degree is in theoretical math, but I was always very interested in actually doing applied math, and particularly using math to make better decisions, especially in this data-driven world. So, in a nutshell, that’s what operations research is, for me – it’s a bit of an unknown field for some people. So the idea is pretty simple. Whenever you have a decision, usually a management decision, and there is data, you can use math to create algorithms to basically make better decisions. So, right now I’m a Ph.D. student at MIT, in the Operations Research Center. And what I do in my research, I basically focus on revenue management when customer choice is important. So, you can think whenever a customer goes to a store, either online or brick and mortar, they have to choose from a selection, and there is a lot of research about how people actually choose from different items available, and that actually impacts revenue a lot. And then, the other type of research that I’m focused on is actually supply chain optimization – so, basically how retailers think about where to keep their inventory in a big supply chain, and how to move it to satisfy demand.
Ciprian Borodescu: That’s a hot topic in today’s Coronavirus crisis and supply chain.
Andreea Georgescu: Yeah. So I’m working with an industry partner through my program and basically, right now everyone is just focused on making sure that the essentials are really available for consumers and doing a lot of work to sort of prioritize those items that are indispensable and making sure that they are putting in all the stores and sort of they are there on time. So it’s definitely a very difficult problem; you know, just thinking of the very classical brick and mortar, there’s always a question of what the demand will be and where it will show up and how you manage your warehouses and even further your upstream supply chain and how much you order, and when do you ship it, where do you ship it? I think it’s actually an interesting distinction these days because I think the main focus right now is that you really don’t want to run out of stock on these very essential items. Whereas in more normal times, you have other problems, where you actually want to run a more lean kind of operations, and you want to put the items in front of people that they need, and you want to have everything available, but you don’t want to keep too much in your stores.
Ciprian Borodescu: Yeah, it makes a lot of sense. But give us some concrete examples of use cases that fall into this category of operations research.
Andreea Georgescu: So, in a very simplified version of the problem, this is called The Newsvendor Problem. The idea is that a newsvendor needs to order a number of papers from the printing, and the question is, how much do you want to order? You know, if you don’t order enough, then you will not meet demand and obviously, you will lose revenue. But if you order too much, you will never sell the papers that you order. It’s called The Newsvendor Problem because specifically for news, you know, you can’t sell the papers in the later days because if you think of a daily newspaper, no one will buy it in the following days. So this is a very classical use case of operations research, you know, how do you think of your demand, of your costs, and your trade-offs? The main trade-off in this problem being the trade-off between not meeting demand – so having lost sales – and the cost of ordering – so paying for the newspapers that you buy, but you’ll never sell.
Ciprian Borodescu: So I did my homework – well, sort of. But I did browse The Newsvendor Problem. But essentially, what I want to understand is if the uncertain demand also refers to times such as the ones we’re going through right now, or this newsvendor model is appropriate for slight variations in demand, not outliers?
Andreea Georgescu: So that’s actually a very good observation about all of these classical operations research models. You know, in the old days when there wasn’t really a lot of data available, most of these models that people worked with started with, “The demand is this.” And it’s usually just a random variable, a distribution that is given to you. Now, obviously, in real life, no one gives you that demand distribution for free. So, actually, a big part of how you would use these models in practice is figuring out what the inputs are to these models. And estimating demand is definitely one of the first things that people need to deal with or estimating whatever other inputs you’re using in your models from real data. So, with regards to your question, I think such a model will be useful these days as well, if you’re using the right demand, which you would have to sort of estimate on these times that you call outliers, as opposed to just using the same demand you would use in normal times.
Andreea Georgescu: I actually think the bigger difference in these times compared to usual times is the trade-offs. I think in these times, as I was alluding to before, what changes is that really, you don’t want to run out of stock. And this is actually common to retailers in general. The cost of not meeting demand usually outweighs the cost of ordering too much by a lot. But in these times, I imagine it’s even more developed, like, you really don’t want to lose demand on some items or the essential items and on the other items, you probably just don’t even carry them anymore. So this trade-off between how much you want to satisfy demand and the cost of ordering too much, usually in this model in particular is really summarized by the service level you want to guarantee. So, if you want to meet 95% of the demand, then you order a given amount; if you want to satisfy 99% of the demand, you order more. And this service level is in practice really set by managers based on intuition about the business, but in a more rigorous formulation is really given by the cost of not meeting demand and ordering too much.
Ciprian Borodescu: Let’s shift gears a little bit. I want to understand what is the life cycle of such a project? Where do you start? What are the usual steps and timelines for these projects until they get to production?
Andreea Georgescu: This is actually one of the differences if I can take a bit of a detour between operations research and something like machine learning. You know, in machine learning, really, the task is already given – you have data and you want to predict something, and the idea is to find the best mathematical models to do that one task. In operations research or for a data scientist, if you want, really the problem is usually a lot bigger and a lot more complicated. There are a lot of missing pieces of the puzzle and it is just a lot more complex. And I think the defining thing in operations research is that we work with real systems – either a supply chain or you can have a hospital with queues or whatever you want to imagine – the idea is that there is a real physical system behind this and we always want to really understand how it works and do something that is implementable in that system.
Andreea Georgescu: So, with that in mind, all of the projects I’ve been involved in really start with the problem definition. And this is something that actually takes a long time. It’s a lot of interviewing with people who are part of that system, who work in that system, from managers to just the people who operate in that system – and I can talk about some other projects that I’ve been involved in. But it’s really trying to understand how their system works, what the problem they’re trying to solve is, what their pain points are. And then understanding what data is available, what are the main constraints – operational or otherwise – and then really formulating a very well-posed problem. And usually, how we formulate these problems is in the framework of optimization. I don’t know if you’re familiar with that. So, basically, what you want to do is to optimize an objective and you have some levers that you can pull, and then you have some constraints that you need to meet. So this is how we usually formulate the problems that we attack. And then, after that, there’s a lot of time spent on sourcing the data, finding it, cleaning it, understanding it, really, and understanding how to model it – this is sort of what I was saying before – like, what do you assume about your demand, for example, and know the inputs in your model, then creating the models themselves, which are usually optimization problems, solving them, and then, the more interesting part, I think. You know, a lot of these very mathematical models give you solutions that are often really not easy to implement. You can think, for example, if you ask me how many newspapers you want to order, I can tell you “Oh, you need to order 3.75.” You know, that’s obviously easy, you just round to four. But actually finding heuristics that are very simple to implement but are very close to the optimal solutions that we find is actually a big part of this project as well.
Ciprian Borodescu: I’m guessing that there are multiple roles involved in such a project, both business probably from the company’s side, and technical from the research side?
Andreea Georgescu: Yeah, so actually, this is one of the great parts about the program I’m in. The Ph.D. students, we kind of end up doing a bit of all the roles, and in this project, where we have an industry partner, we really ended up being the main liaison between the partner and our team, which is usually either one or maybe a couple of Ph.D. students and our advisors who are professors in Sloan MIT’s Business School, and we kind of end up knowing all of the parts and having to navigate through all of them, which is actually why I chose this specific group I’m in. I think it’s very exciting to do all of these things that require very different kinds of skills.
Ciprian Borodescu: That’s super interesting because one of the roles nowadays, especially when building AI products or machine learning driven products, is the PM – the product manager – and I imagine that somebody in your team also has that role. Or is that role kind of like divided amongst yourselves?
Andreea Georgescu: Yeah, so I actually think it’s divided, we don’t have a specific person who is the product manager. It’s usually the Ph.D. student, but our advisors help us a lot, and I think often they have a lot more experience and intuition of what will go and what will not, just because they’re experts in this field, and they’ve seen hundreds of supply chains, and they kind of understand the dynamics better than we do.
Ciprian Borodescu: It’s very interesting because what you described so far, sounds a lot like a day in the life of a startup.
Andreea Georgescu: I’m very happy that you say that. I actually thought that myself when I chose this field. It’s maybe a startup on training wheels, just because we have all of the support of really senior people working with us, which is great.
Ciprian Borodescu: Awesome. What are some of the other challenges? Of course, from a technical perspective, you have a lot of challenges, right? But what have you seen on the business side for companies that are, let’s say, ordering this type of research?
Andreea Georgescu: To my mind, there are a few challenges. One is definitely the data availability and data quality. There are companies out there who have a lot of data and it’s very clean, very comprehensive. But there are also smaller companies that do not. And I think, really, you know, what you put in is what you get out. So, if there is no real data to drive insights, then there’s only so much you can do. I think a very big challenge is implementability. So, for example, I did a project – it was a shorter project, six months – with a local hospital in Boston; so people would have appointments scheduled, but they would just not show up. So what happened is that the providers actually had to schedule a lot more appointments in a day, just so that they see how many patients they need. But then, you know, they had some days when three out of the 12 patients scheduled would show up, some days when all of the patients would show up. And then, you know, there would be three hours overtime. You can predict whether they will show up or not. So, you know, on a given day, you can schedule, let’s say, eight people that you’re sure will show up and four people that maybe won’t show up and one of them will. A very simple problem mathematically. What it turns out, though, is that the hospital itself didn’t have the ability to implement such a solution, just because they couldn’t really implement this kind of differential treatment to the patients. Just because, you know, appointments are made through a call center that’s super busy, and it just never worked. They could never include this step in the appointment system, to check what kind of patient they’re dealing with, and all of that.
Ciprian Borodescu: So this is really interesting because while there is a technical solution to the problem, the solution was not feasible for their context.
Andreea Georgescu: Exactly. And it’s not feasible, really, because of the physical system that you’re working with. And so, what we did when I was around for this project, the fourth or fifth time, we tried to target the providers themselves, and how they create the template of appointments in a day – they could put some pictures of the patients in there that will automatically show up in the interface that the schedulers were using. And in that way, there was no extra step for the schedulers who were overloaded and couldn’t add an extra step; you would just make the provider when they set up their calendars – you basically say, “In this slot, I accept only this kind of people, in this slot, I accept only this kind of people.”
Ciprian Borodescu: That’s interesting. That’s quite an elegant solution, actually.
Andreea Georgescu: It did take five months of understanding why stuff didn’t work and how you can go around it.
Ciprian Borodescu: So in your line of work, you’ve seen big companies that have the data which was clean, but you also worked with companies, probably smaller that were just beginning the data collection process. What are the characteristics of companies that are best suited for this kind of work?
Andreea Georgescu: So I’ve actually only had two industry partners – the hospital I mentioned and the US retailer. But I do, from all of my colleagues that they worked with a lot of other companies. Honestly, I’m not sure. I think it really depends on the kind of problem you’re trying to solve. Like, for some problems, just a little bit of data is fine and it’s really about people’s willingness to experiment and to implement ideas. For other problems, you really do need a lot of data to make an impact. So yeah, I’m not sure there’s really a recipe. So, for example, one of the colleagues in my group did a project with the grain markets in India. And he helped them create an auction, a bidding system that would actually make information about pricing more easily available to all the farmers, so that the prices would actually increase and be more uniformly distributed across all farmers, and thus, reducing the variability in the revenue farmers get for the same kind of grain. And in that, I think the data he had was really not that sophisticated, but they managed to have a lot of impact there. So, yeah, I think it really depends.
Ciprian Borodescu: And I think it’s also a matter of going from a proof of concept or from research, there’s also a matter of leadership to go from that point into a real product. Right?
Andreea Georgescu: I think that’s definitely true. So, a lot of the work we do is theoretical, but a lot of the work we do is very much applied. I really think that our advisors play a huge role because they are very senior professors at Sloan, who are very well-respected, and they manage to drive these changes and impact institutions.
Ciprian Borodescu: And out of curiosity, how does this work? Is it that an enterprise or a company comes with a problem to you, or is it the other way around? Or is it both?
Andreea Georgescu: I think what happens is sort of half-half. Companies come to us and then they either have a very specific problem in mind, or they have a specific area that they feel is not functioning as well as it could, and, in that case, that problem scoping part of the project is really a lot more involved on our part.
Ciprian Borodescu: And the reason I’m asking is, for a company, if this research is something that they can do in-house, or externalize it to researchers, such as yourself, or other companies doing this kind of work.
Andreea Georgescu: I think there are plenty of consulting data-oriented companies. I know, for example, McKinsey has a special group that is more data-oriented. And I’m sure there’s tons of others. And definitely, companies also have a lot of in-house data scientists. I think, though, one of the benefits of working with us, you know, we’re kind of a fresh perspective – which would be true with consultants as well – just because we’re not part of the company. And then, as I said, you have these very senior professors working, you have students like myself who are pretty hungry and motivated.
Ciprian Borodescu: Yeah. Now, you’re no stranger to the startup world and I wanted to ask you, what are some of the areas you’re mostly interested in, or maybe you’re following these days?
Andreea Georgescu: One of the areas that I’m super passionate about, but I’m following, really, as a hobby, because I haven’t seen any progress done in that direction is really media and news publications. You know, I feel there’s just too much information there for people to process. And I think that maybe the mechanisms used that make some content available and bring some content in front of viewers’ attention are maybe not necessarily in line with what I would want. But I actually haven’t seen much progress in that direction and I can’t say I’m working on it either. Although I do know of a friend at MIT, who is starting to work in this direction, to basically change how platforms like Facebook or other social media platforms, how they put content in front of viewers and sort of think of more fairness or welfare measures that could drive how we think about all of this influence that these platforms have on knowledge distribution. But I think it’s just starting to ramp up.
Ciprian Borodescu: And it’s early days. Yeah.
Andreea Georgescu: It’s early days, yeah. Other than that, I think any kind of personalization and predictive and optimization based on that. Predictiveness is something that interests me. I don’t actually have a preferred industry, I would say.
Ciprian Borodescu: Okay. You and I talked a bit about your plans for, I believe, the second half of this year, which I found very interesting. I don’t know if you are in a position to share that publicly, but if you are, what can you share about the next adventure?
Andreea Georgescu: So, given the current crisis, I don’t think any of us knows where we will be in half a year but yeah, currently I’m planning to do an internship with Airbnb as a data scientist for the summer, which I’m super excited about and I hope it will still happen.
Ciprian Borodescu: Yeah, Airbnb is an awesome company and I know they are doing a lot of stuff around data science and, of course, machine learning and stuff.
Andreea Georgescu: Yeah, so that is something that always interests me. So, in general, two-sided markets – and not related to Airbnb – but auctions in general and how this sort of complicated dynamics play out is something that interests me a lot.
Ciprian Borodescu: Honestly, I learned so much from you. Thank you for taking the time to do this episode. I don’t know if you noticed, but we started a few weeks ago an open-source initiative to help online retailers to optimize their budgets using AI and ML during these strange times and let it be known out there publicly that I was inspired by our very first conversation almost a month ago to kick-start this initiative locally, at a small scale. So, once again, just wanted to tell you from the bottom of my heart, thank you for sharing your wisdom with me and with us.
Andreea Georgescu: Thank you. It was a pleasure. And I can’t wait to see what else you’ll talk about.