Latest / The Laravel Podcast / Laravel Excel, with Spartner's Arne Schoenmakers & Patrick Brouwers

Transcript

Laravel Excel, with Spartner's Arne Schoenmakers & Patrick Brouwers · The Laravel Podcast

• Matt Stauffer: Welcome back to Laravel podcast, season five, where every single episode is not about a topic, but about a package. And the package we're talking about today is one of the two or three most popular packages in the entire Laravel ecosystem, which is Laravel Excel. My guests today are I'm going to butcher their names but I do my best. Arne Schoenmakers, the founder of Spartner, which is the company behind Laravel Excel and Patrick Brouwers, the software architect at Spartner and the lead developer on Laravel Excel. And for those of you who don't know, Spartner is the new name of a company that we all are more familiar hearing as maatwebsite. So you'll probably see maatwebsite slash Laravel Excel everywhere, Spartner's their new names. That's why we're talking about it. So guys, welcome to the show. Thank you for being here. And could you each just kind of say hi and tell the people a little bit about yourselves? Arne Schoenmakers: Yeah. So hi guys, I'm Arne Schoenmakers. So as you just told the audience, I'm the founder. I'm actually, let's say a founder that really started as a developer himself. I developed during my studies, made some money actually paid for most of my student years by developing stuff with BHP. And when I was finished with my study, I actually studied international business but was kind of a computer addict. So taught myself development and also did some courses. But when I was finished, I just decided to start full-time as a developer for myself, on my own, developing stuff for random companies that I just walked into in the city. And that's 2004. So me 42 right now, it's actually 18 years ago. So yeah, it's been a while and now we've grown to a nice development team of which Patrick is one of the architects. Matt Stauffer: Right, Patrick. Patrick Brouwers: Well, hi, Laravel community. I'm Patrick and Brouwers is my last name, which is indeed very difficult to pronounce. If you're not Dutch. You can hear that the rolling "R" I try to pronounce as good as possible. I've been working at maatwebsite, which is now a Spartner for a bit more than eight and a half years, which was also my very first job. And I've worked with Arne ever since. And since very early we started working on Laravel Excel since I joined. Matt Stauffer: That's awesome. Well, thank you guys so much for being here, it's a total pleasure having you, and I know we'll kind of get to know a little bit more about you and the package and everything like that, but let's start out with the original pitch, the elevator pitch for Laravel Excel. So if somebody's never heard of it, what does it do? So Arne Schoenmakers: Actually the pitch is on our website, laravelexcel.com, supercharged Excel exports, and imports for Laravel. But it actually started out as exports only because that's the main feature that we wanted to highlight. Because it started actually right after us getting introduced with Laravel back in 2013, I guess we started on Laravel 4, 4.0 a beta version. And I think it was somewhere near the summer where we started it for a project. And we were actually doing our new at that point maatwebsite website. And we're thinking of let's make demos. And we actually had a client that producing Excel in a project. So we were working on that and then we thought, and actually Patrick came up with a great idea. Let's do a package. Arne Schoenmakers: And that was, I think, the birth of Laravel Excel as a package, we thought let's make something as a nice demo for our website ourselves, that you can just fill in some random numbers and a spreadsheet and press export, and then you'll get an Excel. And that would be really nice for clients to see how easily and powerful our work can be. But then for the developers, we thought, "If we make it a package and anyone using Laravel can do that by the click of a button from their database or whatever." And that was just the first step, just easily creating Excel, having something magic happen within one click or one line of code, ideally. And adding that feature, which almost any content management or ERP system that we built has. So yeah, it's just really neat and, and, and Laravel didn't really have anything, so we thought that would be great. Great fit. Matt Stauffer: Yeah. So, if we talk about the main job that it solves, I mean, you just said it very quickly. If you want to export structured data into an Excel, so XLS or XLSX, this is the way to do it. And it's super simple. It's almost to the point where when you're using it and you're like, "Did I need this package?" But of course you did, because if you do it in your own it's miserable, but it makes it so easy that you're like, "Oh, it's just a single line of code." It's just kind of like, "Why should this not be this easy?" So I'm very grateful to it because when I've needed it, I've had to do Microsoft connected encoding manually before with things. And I just want to quit my job and go herd sheep. Matt Stauffer: So the fact that you made it so easy for me to not think about that is pretty delightful. So you did just tell me the story, the creation of the package, which I think is really interesting because I think, I don't know if I've ever heard anybody say "I created this package to show what you can do." So that's a really cool idea that you just want to be able to say like, "Hey look, you might not all know what our capacity is or what Laravel's capacity is, but we're going to show you a really cool thing we can do." And the fact that it's tied to them just typing in random numbers and data shows that this isn't a pre-made thing. We just made it. So that was a really cool idea that you guys did that. I love that you open sourced it too. So it's not just proprietary to the two of you or to your team. So that's very cool. Arne Schoenmakers: The fun thing is we started it as this random idea that evolved and evolved. And of course, everyone wants to give back to the community and grow it. And at some point you really start to put effort into that. But first it starts out at just a brainstormer, an idea that comes from a certain need or something and a certain idea and just wing it and put some like minds and energy to it. And then you have something and you put it out there and it's a package and you don't know anything about success, but then because it becomes a success, or it got a lot of downloads at least, you start to think should we, should we actually incorporate this on a day to day basis or weekly basis or monthly basis? How much time can we spend on it? What is the value of that time? Arne Schoenmakers: Because I used to be developer only working as developer, but currently I also have to run the company, have less time for development. So I have to think about these kind of things. What do you do with a open source package when you also have clients worried about how much time you can spend for them on a project? I think any developer doing it by himself has that challenge, but as a company as well you need to think of things that we did. Like, shall we allocate weekly training hours? Anyone can do anything on any package or just try some new stuff out that might be valuable for a client, but might also just be something to do and something to train yourself. And it might be really good to do it from a recruitment purpose that you meet developers and interact with them on challenges that they have, and they find you or whatever. Arne Schoenmakers: And all those values, I think, came into the idea way after the fact that we actually just started it. We started it and then we started thinking about it and then we thought, "Let's just do this frequently. Put more time in it, try to get it evolved." And you have these intervals that you put some more ideas into it, and more time in certain periods than other. But yeah, it's been a really great ride, currently 38 plus million downloads. Matt Stauffer: I love it. Arne Schoenmakers: It's amazing. It's such a high number. You can't even imagine it, how many developers might be using it, but we're just so grateful that that happened just ongoing and yeah. Who would've imagined when you just- Matt Stauffer: That's awesome. And I do, of course I identify with that because at Titan we make packages and sometimes they're just there for people and sometimes they have business impact for us. And I know that we'll have Spotsy on again in the future or, well, we know we don't have them on yet in this podcast. And the same thing, they're a business that's making packages. That's just trying to figure out the interaction between your business and your package. So I love hearing some of the thought process as you, as an owner of a company have behind. Because we have the same things. We're like, "We'd love to give you all as much time as possible." So for us it's every Friday people can work on packages and stuff, but even then, what can we do to make sure that everyone can work on our packages? There's still a lot of really interesting questions and challenges about it. So I appreciate you sharing some of those things there. Arne Schoenmakers: Yeah. And what we also thought of these things, because we're not just saying you can only work on our packages. We thought if you have ideas and just like Laravel Excel started from an idea, just write it down. Try to structure it a little bit so that you're not just browsing some kind of website and just reading, but try to target something. Do you want to do something with UI or do you want to learn something, whatever? Make something for Tillwin or contribute to a package that somebody else did? Or do you do something with Laravel, PHP or any kind of technology? And we just try to challenge our people and ask them, "Okay, if you have an idea, just spend some time on it, allocate some time." Actually Friday is a really good moment for us as well to do this kind of things, because people are almost into the weekend. It's nice. And yeah. And I think maybe that's the way forward to support these kind ideas and get these new ideas out there as well. Matt Stauffer: Yeah. And I love the idea, and I know we need to move forward, but I like the idea very much of supporting the ideas that are coming from, not just the leadership. Our best packages have not come from me. I don't know if any of our goods packages have come from me. So yeah, absolutely. I guess there's one, but yeah. Okay. So let's keep moving on. Because I know that Arne and I could just talk business all day and this is supposed to be not about business, but I'm loving it. So obviously the installation steps are just going to be a composer require, but are there any key set steps or dependencies that you want to make sure people have heard about to make sure when they're using it? Patrick Brouwers: Well, maybe it's good to take a step back, and what's the backbone of the package. Because of course we don't handle all the Excel generation stuff ourselves. In the back end it's all powered by PHP spreadsheet, which we have been using since the very beginning before the Laravel 7, or the PHP 7 release, it was called PHP Excel. A lot of people have heard of it. So that's still powering our backend and we're very happy with their contributions as well. But I think when I initially started working on this demo thing that Arne described, it felt different than the entire Laravel philosophy. And that's what I was missing. I was so enjoying all these fluent things that Laravel offers. I just had the feeling that's what I want as well when I want to work with Excel because Excel, indeed, like you say, it's not the most fun thing to work with usually. Patrick Brouwers: And it did evolve from now we're at 3.1, which is even more doing the work for you. So talking about key dependencies, we have two generation commands, which is very similar to making a control or making a command or making a middleware or a job. You can create an export and you can create an import. You can pass some parameters. You can say I want to use a model as a basis, or you can use a collection or an array. Because I think the basic concept of every export is your data source. You indicate what your data source is. That can be a query, that can be a model, an array, a collection. And you can basically determine yourself how you want to load this data. Patrick Brouwers: The main use case, I think of the package, of course, because it's a Laravel package is to generate exports from an entity, from a model, from an eloquent model. So we made hat as easy as possible. So I would say that's the main first step. If you install a package, you will create an export for maybe your user's database or your orders database or your payments table. And basically all you have to do is run that command, pass the model in, and it will create an export clause for you. And in your route or controller, wherever you want to put it, you call the export and it will just make an expert with all your database columns in that export. And of course the next steps are then to configure your export. Matt Stauffer: Sure. Customize it however you want the data end up. Patrick Brouwers: Filing, data formatting, mapping, that kind of stuff. Same goes for imports. It's the same story. You import something to a model or you import something to a collection and you handle it yourself. Matt Stauffer: Okay. So that's the basics of what most of us are going to interact with on a day to day basis. We're going to say, "Here's the button where you can take an Excel file that is structured like this, and we're going to validate it. And then we're going to bring the data in, or here's the button you press, where of all the ones you have selected, we'll export them out to an Excel." And that makes a ton of sense. Are there any lesser used features or cool tricks or optimizations or tweaks that people could do that you don't know if everybody actually knows about that you'd like to share? Patrick Brouwers: Well, one cool thing that has always been in the package and that we develop actually for a client project is a way to indicate, or to name certain cells when you're doing an import. Because we had this situation, I think Arne remembers this as well. We had a, how would you describe it? A weight loss company, a company that helps people lose weight. They asked us to build a new system for them and they'd always been working with spreadsheets and they wanted to have all that data in the spreadsheets. But these spreadsheets were very structured. Like in cell A1, it would have the name. So it was a bit, but it was also possible that the structure was different, they had three, four versions of the same spreadsheet. And we would be able to identify like, "Okay, these 10 are type A, type B." Patrick Brouwers: So what we would do to keep the main logic, the Laravel logic, the same would be to map these cell else to specific keys. And then we would just get an array with, this is the name of the person. This is the weight, this is the weight loss. Without needing to have four different handling of the import. We just had to map which cell is which data. And I think that that's something that not a lot of people know, but could be very useful for a lot of people. Matt Stauffer: Yeah. I certainly didn't know that. And even if you didn't have multiple of those, I still think the idea of being able to just do a one time mapping of column A is this, column B is that, even if it's not three different ways of it still will probably make it much easier. And like you said- Patrick Brouwers: Well, this feature is less about columns, more about cells. Because columns is a different story. Maybe something I can share later when we could possibly talk about future things of that Laravel Excel will add, but this is specifically about cells. So you wouldn't indicate a complete column, but you would really like get a certain cell from the middle of the sheet maybe is Z10 or something. Yeah. And get that value into an array. Cool. Matt Stauffer: That's great. So, I mean, we're actually right onto the next question, which is the development roadmap. So could you talk a little bit about some of the plans you have for the future, including the one you just referenced? Patrick Brouwers: Yeah. We do have some 4.0 ideas because I think these ideas that we have are quite new and add a lot of excitement to the package. And they mostly evolve around data, the way you structure your data. So there's not a data source, but like the column thing we talked about, like I mentioned, reacting to you. Which basically means that it looks very much like how Nova works. You indicate that, Okay. I want a tax column in column A, which is the name. I want a date-time object. I want a number, or maybe money." Because I think that is the biggest pain point as well currently if I have to implement an export on import, it's still tedious to define I really want this to be a number in Excel and I want this to be formatted like this. Patrick Brouwers: And that this is something that we want to make a lot easier in the future, in this 4.0 release. And that's the same thing as it works for exports, it will also for imports. You can just say, "So I want to export a date-time, but I can also import a date-time." Which makes it lot easier than having to do this all by yourself. Parsing the date, looking for the date structure, parsing to a carbon object and inserting the database that will all be handled by the package then. Matt Stauffer: So of when you're exporting, so if I were to say, I'm exporting this particular cell is formatted like date-time, are you actually with this goal being able to send whatever command Excel knows to actually treat it that way? Because I was thinking, "Oh yeah, we can send it as whatever data we want, but I assume that by default Excel will just look at all them as a string until you actually hit the format as whatever." So you're actually talking about being able to send whatever command Excel needs to say format is this. That's cool. Patrick Brouwers: Peach spreadsheet does some auto formatting. It tries to see, okay, this is a date-time. So we will do some auto formatting on the logic. But when dealing with very big imports, we disabled certain features to make it more performant. But it also means that Peach spreadsheet has a lot less to work with because it doesn't have styles. And a lot of the information comes from the styles in this case. So by letting the developer define that this is a date-time column, we can make the assumption already on a very early level that, okay, whatever comes into this column, this should be a date-time. So we will try our best to parse it as a date-time. Matt Stauffer: That's awesome. That's very cool. Any other development roadmap plans that you want to share and then let's transition from right into that. Where do you want help with any of that? And we can talk in a second about, do you want to make other kind of help, but in terms of that feature you just said, are you guys looking for help or you already have the plans and also, are there any other plans that you have going forward? Arne Schoenmakers: Yeah. So we have this one thing that we've in the pipeline because Patrick just mentioned the performance issues some people have. And we actually also encountered that and helped clients with that, that reached was commercially with implementation on their thing that was stuck or couldn't perform. So we actually have something in a pipeline called Laravel CSV so that we can actually get more to a base bone baseline version of data structure that can handle way more. And way more data than the Excel would to be able to do because Excel simply has a limit of the amount of rows that it can handle. Arne Schoenmakers: So yeah, that is something that we're thinking about how to get that out there. And if we want to do it under the flag of Laravel Excel, just as a feature or as a separate kind of package that is more oriented towards CSV. So we're thinking about that, how to do that. But yeah, that might also be something that people can contribute on by either replying or doing something on social media to tell us how they would prefer to have that CSV performance feature, what they've encountered themselves. Because we're still contemplating on what would be the best practice to say okay, maybe in some situation you don't want Excel, but you actually want raw CSV structures as a data export that might also really help. Matt Stauffer: Yeah. And I totally understand what you're saying. You might be able to actually just say, "Hey, if you're running into performance issues on this particular one, switch this toggle. All the interface will still be relatively similar, but you're going to be doing with CSVs." You have to tell the people how to make an CSV, but all of a sudden you're going to get all this performance benefits without having to switch to an entirely different package. Because right the League CSV package is fantastic. But switching from Laravel Excel to League CSV would not just be a toggle. It would be ripping out your Laravel Excel code and then putting in your League CSV code. Whereas if it something that's built into Laravel Excel or a parallel package, it might make the switching back and forth process a little bit easier. Which makes a ton of sense. Patrick Brouwers: Yeah. Some something that we have played with in the past, because we've had some commercial support clients for this in the Laravel community for which we developed a similar package. And it was really intended at being a drop in replacement for Laravel Excel so you could just keep everything in your export class. You only have to change the name space and the class name and you get all the same benefits. Because my opinion on big exports is that you probably don't need an Excel if you need a big export because you won't do styling, you won't do formatting. You just need the raw data then. Matt Stauffer: Yeah. Totally agreed. I love it. And the same's probably true for the imports as well. When the people are uploading something that's so large that won't process well as Excel, it's probably not come from a human being that was creating in the first place. Patrick Brouwers: Exactly. Yeah. Arne Schoenmakers: Yeah. Matt Stauffer: Okay. So the one thing you mentioned for help and support is for people to get back to you and just suggest their ideas about what they think the best next steps would be there. Should it be a part of Laravel Excel, should it be a separate package? Do they have any other requests? Is there any other support, whether it's programming or a Patreon or something like that where people say thank you for Laravel Excel, what does it look like for them to say thank you? Arne Schoenmakers: Oh yeah, actually we call Laravel Excel postcard wear. So we would really appreciate postcard. We have a lot of postcards here at the office. We of course sometimes takes pictures of them and they're also in our short movie that we just released on the rebranding from maatwebsite to Spartner. But yeah, we really appreciated. It's just fun gathering them from all over the place. And we have actually received a lot of cards to date and would love to get more of course. Other thing, contribution wise, I think the community is already shown that they're really growing in the quality of, let's say remarks that they're giving. Because of the way that people currently use GitHub and those kinds of things, maybe Patrick can say a little bit of the background how that was improved. But I think the support tickets that people requested used to be maybe a bit of a pain and nowadays they're way better. Patrick Brouwers: Yeah. I think that really involved, of course, if a lot of people are using it, a lot of people will run into questions and problems. So I think there has been times, especially around the 3.0 Release that yeah, maybe 10, 15 tickets per day. And that's just not, you can't handle that on your own every day because it takes up a lot of your time. And of course, I also have my day job to work for our clients. So I can't spend all my time on replying to people. And I think around that time GitHub released a feature called the issue templates and that really helped to help people structure their question. Because if it's just a question with a base it in piece of code that that takes a lot of time for me to parse through and understand what they're actually asking. By asking a few simple question, that questions that really helps people to understand their own problem. And I've even heard from some people on Twitter later that it was like a rubber ducking kind of thing. They would understand their problem and then think, "Oh, I don't even need to-" Matt Stauffer: Don't even need help anymore. I love it. Patrick Brouwers: Yeah. And I think last year, GitHub even made the issue templates even better. Now it's like an input form. So you have to fill in the fields, you can't delete the issue template anymore. Plus we split it into box. And in the discussion tabs, you can ask any question you want. And discussion tab is a bit more community driven, everybody can of course help people with problems they have or questions they have. Or we really try to get, keep the buck, bucks tapped, the issue issues. Really centered around things that really don't work. Something that needs our attention or someone else in the community who can PR it, because we thankfully also get a lot of PRs of people that need something very specific for their situation. And we see how it incorporates into our idea of the package and we release their contributions to the next release. Matt Stauffer: That's awesome. Well, I think it's super clear what it does and how to use it and what's coming next. And what kind of support you guys need. Is there anything else you'd like to cover before we call it quits for today? Arne Schoenmakers: No, just, I think it's been fun talking about it, but the topic we love doing it still gets us enthusiast to do more in the future. Maybe have new packages coming out that also would get a similar success, but if not then still it's fun to do these kind of things. And yeah, it's been a great ride so far so. Matt Stauffer: Awesome. Super. Patrick Brouwers: Yeah. So something that I personally really like about the Laravel community is this synergy between the package developers and the package users, because we also started using Laravel and using some packages and that inspire us to make packages. And I think that us providing packages hopefully has also inspired other people. And that's something that we benefit again from, so yeah. It's like an ecosystem keeping itself alive by providing packages and using packages Matt Stauffer: Yeah. And collaboration, everyone's trying to help. And each of us who's helping other people are also benefiting from other of people's help. So I love that. That's a really great insight. Arne Schoenmakers: I would also ask developers to actually tell their potential bosses to say there's value in actually working in job time on a package. Like there is value and there's a lot of value for your people. There's value for the community that will pay itself back. There's value for company as show of skill, there's so much value. And I think several companies already do that fantastically. I think it could be a lot more that will help the ecosystem help Laravel grow even to a higher level that it is currently. And I think there's so much value and I think the developer seeing this should just drive with this colleagues and say, "Okay, let's just get some time every week to do something, maybe one in two, once in three weeks." Just something during your day job, it would just be great to have a frequency, to get more people doing this. Matt Stauffer: Yeah. Yeah. And as business owners, we can attest that it's not free, but also there's value that comes for it. And you mentioned there's value to the employees, they're learning and they're growing and they're happier. There's values to the community because they're getting free work and there's values to the company because like you said, people can see our capacity as programmers. They can see our mentality of wanting to help people. And our name gets out there. Every single time somebody installs something that says maatwebsite slash whatever, that maatwebsite now gets embedded in their brains a little bit. And so there's also a marketing benefit that comes from owners being willing to make this time available. So I'm really glad you said that, Arne Schoenmakers: But then the problem is if you rebrand Spartner, then you have a problem. Matt Stauffer: Good news is they can just go Google my website and they'll still figure out who you are. Arne Schoenmakers: They're going to figure it out. Patrick Brouwers: Let's hope. Matt Stauffer: Right guys. Well, it was a ton of fun having both you on today. Thank you so much for this package. Thank you for your contributions to the community and thanks for hanging out today. Patrick Brouwers: Thanks Matt. Arne Schoenmakers: Thanks. Matt Stauffer: All right, everyone else. We'll see y'all next time.