Equinix is the world’s digital infrastructure company®. Digital leaders around the world leverage Equinix’s reliable platform to seamlessly integrate and connect core infrastructure at the pace of software. In addition to building innovative solutions that help organizations scale with agility and deliver world-class experiences, Equinix also continues to make progress on its sustainability goals. Founded in 1998 in Silicon Valley, Equinix has grown exponentially in the last few decades and has more than 250 data centers around the world. 

So, in order to understand the scale, challenges, and tech stack involved in hiring at such a scope, we had a conversation with Swetha Harohalli Ravikumar, University Recruitment Specialist, and Dennis Wilfred, Talent Acquisition Lead – Global IT, at Equinix.

Their insights paint a comprehensive portrait of Equinix’s journey in India – in ensuring they not only create positive experiences for candidates and hiring managers, but also ensure the quality of candidates selected meet their global bar. 

• What were the primary challenges Equinix encountered in attracting and retaining talent before implementing HackerRank Engage? Can you provide some context on this?

Swetha Harohalli Ravikumar: When I first joined Equinix, the company was in its early stages of hiring interns and graduates in India. Through years of collaboration with universities, we had recently welcomed our most significant batch of interns and graduates.

I observed that since it was one of our initial forays into hiring this demographic, we needed external support to liaise with some of the universities. Given our rapid growth in India, especially with Bangalore emerging as a hub for Equinix in the IT space, it was important for us to establish stronger ties with academic institutions. 

The challenge was that Equinix, being more of a B2B enterprise, was not well-known to the student demographic. This posed a hurdle when engaging with students, many of whom had set expectations when considering their placements. Our goal was more than just recruitment; we wanted to create an ongoing dialogue with these students, many of whom we hoped to bring on board for internships in the coming years.

Dennis Wilfred: Equinix’s foray into India began with the acquisition of a company. Following this acquisition, we decided to establish our global IT development center in Bangalore. Given this strategic move, our immediate priority was to seek out talent with diverse skills to help us build our IT organization in the region.

However, our entry into the Indian market came with its set of challenges. Notably, while some segments of the industry, like platform or data teams, were familiar with Equinix, many in the developer community weren’t. Considering our focus on the IT and developer side, building brand awareness became essential. 

Further complicating matters was the fact that our hiring managers, given our acquisition-driven entry, were predominantly from regions like Singapore, the US, and Europe. Acclimating them to the nuances of the Indian job market, its talent landscape, cultural differences, and challenges such as candidates declining offers, required deliberate effort on our part. 

Another specific challenge we grappled with was the considerable amount of time hiring managers were investing in initial candidate evaluations. More often than not, every candidate profile we received looked promising, leading to a high number of interviews. However, these extensive efforts weren’t translating into successful hires at the rate we anticipated. Recognizing this inefficiency, we saw the need for a platform or tool that would assist in evaluating candidates, allowing us to present only the top-tier profiles to our managers.

Moreover, we wanted to enrich the experience for both hiring managers and candidates. In the past, a manager might speak to dozens of potential hires, with only a fraction making it to the final stages. We envisioned a system where managers would engage deeply with a smaller, more curated pool of candidates, dedicating time to elaborate on the role, our tech stack, the problems we aim to solve, and the impact we strive to create. This change in approach not only improves the hiring process but also provides candidates with a clearer understanding of what Equinix aims to achieve in India.

It was at this juncture that we considered partnering with an organization that could address these challenges. Among the solutions we evaluated, HackerRank emerged as a compelling option. Our primary goal was to streamline the hiring process, ensuring quality interactions and delivering top-tier candidates to our hiring managers. With HackerRank, we were able to provide candidates with real-world coding challenges, offering them insights into the problems we’re tackling.

Today, I’m proud to say that our hiring process in India has matured. Almost every hiring requirement undergoes this refined assessment. We’ve even introduced this method in Poland and are piloting it with a few managers in the US and Singapore. 

• Could you provide insights into the types and numbers of roles Equinix is currently pursuing? Specifically, the job families and developer roles. Are you still actively recruiting for positions in software engineering, AI, ML, and cloud? A brief overview of your hiring objectives would be immensely helpful.

Dennis Wilfred: In 2022, we filled around 300 positions. This year, we’ve successfully closed around 80 positions in India and have another 30 to 40 on the docket before the year closes out. Our estimate is that we’ll conclude the year with about 120 to 150 new hires.

Our recruitment spans various job families and levels. We’ve ranged from hiring principal architects to software engineers, ensuring we have a diversified skill set on board. This year, we also initiated several Centers of Excellence (CoEs) in India. For instance, we introduced a platform CoE focused on predictive modeling and enterprise planning, among others like Oracle CPU and Siebel. Another critical addition was the Business Systems Analyst role under our enterprise planning and management segment, which plays a pivotal role within Equinix.

While our numbers might seem modest, our emphasis this year was truly on quality. HackerRank played an instrumental role in ensuring we identified top-tier talent. The establishment of these CoEs in India signifies the trust and confidence our business leaders now place in the Indian talent pool. We’re also proud to announce that we recently hired our principal architect for our ServiceNow domain under the WebCut organization, further solidifying our technical capabilities.

• What were the main factors that led you to choose HackerRank over the other competitors you looked into? 

Dennis Wilfred:  One of the primary reasons we leaned towards HackerRank was its global reach. Your presence in multiple regions stood out.

Another compelling factor was the breadth and depth of the skill stack and the library of assessments you offer. We didn’t find such a comprehensive list with any other organization. Some catered mainly to early career individuals or newcomers to the field, while others, relatively new in the industry, simply lacked the types of assessments that you possess. The flexibility that HackerRank offers in creating assessments wasn’t matched by any other organization.

The CodePair feature was another strong selling point. We wanted to move away from mere multiple-choice questions to a more interactive and reflective assessment system. It’s not just about ticking boxes for us, and the same goes for candidates. They’re eager to understand and solve actual problems, which is evident in the challenges we present. This approach was influential in convincing our leaders and other regions, like Poland, to adopt this method. They were more inclined to see an issue that mirrored real-life scenarios rather than just theoretical questions.

Proctoring, too, was a key feature that we valued. In sum, all these factors combined made HackerRank the ideal choice for us.

Swetha Harohalli Ravikumar: Though I wasn’t part of the initial conversations, since I wasn’t with Equinix when we initiated the partnership with HackerRank, my engagement with the platform made it evident why it was chosen. It was an obvious choice, given the exemplary features and adaptability that Dennis elaborated on. The checks and safeguards embedded in HackerRank are commendable.

It’s not just about ticking boxes; it’s about HackerRank’s proactive approach in addressing emerging challenges. This is especially vital when considering the vast audience of new and early-career individuals. These students are sharp, always exploring ways to get an edge or bypass the system. HackerRank’s continual technological innovation to preempt and address such integrity issues underscores its value to us. 

• Regarding your engagement, could you provide some qualitative results or specific success metrics that illustrate the benefits you’ve experienced with HackerRank? We’d also love to know why you believe it has been effective for you.

Swetha Harohalli Ravikumar: One primary benefit was the multitude of touchpoints we managed to establish with students. While we spearheaded the marketing efforts and outreach to universities, HackerRank insights about registration rates, email open rates, and click-throughs provided crucial data about our outreach effectiveness. It allowed us to discern which colleges were more active and which ones required renewed effort. 

The touchpoints were instrumental. For example, out of 1,400 registrations, we witnessed participation from over 650 students. Traditionally, such a high conversion from registration to participation is rare, which indicates the effectiveness of our engagement strategy.

Moreover, post-challenge, the support from the HackerRank team was commendable. Their timely assistance, clarification of back-end processes, and overall support greatly impacted our evaluation process.

Quantitatively, the sheer volume of individuals we reached and engaged with was impressive. On a qualitative note, the evaluation metrics and subsequent discussions with engineers provided clarity about participants’ performances. This has helped us identify potential candidates for future internships, ensuring a pipeline of quality candidates. Both in terms of volume and quality, our experience with HackerRank was immensely positive and metrically sound.

Dennis Wilfred: The preliminary feedback has been overwhelmingly positive. We’ve even introduced it to our brand marketing team, expecting them to delve deeper and explore its full potential. Moving forward, our experiences and the feedback gathered have all been positive, and we’re eager to tap into it more intensively. In fact, one of our immediate plans is to initiate an internal hackathon, different from our usual external ones. 

• Beyond the candidate management features and visibility metrics you mentioned, were there additional benefits you noticed with “Engage”? Specifically, its impact on branding or initial impressions and experiences when Equinix began utilizing the platform.

Swetha Harohalli Ravikumar: Over the course of my career, I’ve worked with several vendors to orchestrate hackathons. My experience with Engage has been distinctly positive, distinguishing it from others I’ve used before. Its most commendable trait is the platform’s intuitiveness. Unlike other platforms where navigation might pose challenges, Engage is straightforward. After discussing with our marketing team, I found it simple to incorporate their inputs, which allowed us to mold and modify our setup to align perfectly with our vision.

The time to set things up was also a very positive surprise. Within a span of just a day or two, the entire setup was operational. The HackerRank team gave a number of constructive suggestions, which helped us further improve. It’s not just the setup phase that’s efficient; the entire user journey on Engage minimizes unnecessary complexities. With past vendors, I often found myself stuck in a constant loop of communications for even the most minor alterations. Engage, however, allowed us a degree of autonomy that reduced this back-and-forth, allowing us to effect changes without being overly reliant on external support.

The structure of the platform, from the initial registration to intermediate touchpoints and finally to the challenge, is very intuitive and easy to set up. This systematic approach helped us in establishing constant communication with candidates.

The depth and breadth of insights offered by HackerRank tests, from the granular details like time metrics to a broader overview of participants’ skill levels, are unparalleled. For any organization, ensuring a balance between quality and quantity is pivotal. While catering to large participant numbers is essential, it should never come at the cost of quality. Engage ensures we never have to make that compromise. The platform not only aids in reaching out to a vast audience but also ensures that the engagement is meaningful and the results are trustworthy.

Dennis Wilfred: A significant advantage for us was the reduced need for hands-on management. In my prior organizations, we spent countless hours assessing and sorting through candidates. Thankfully, with HackerRank, most of that heavy lifting was done for us.

Our main responsibility shifted towards branding and outreach. It gave me the opportunity to focus on key colleges, especially those we’ll likely return to for intern recruitment. This freed-up time allowed us to foster deeper relationships with the coordinators and better understand the kind of students we wanted to engage. It’s about cultivating an environment and rapport that will benefit us not just now, but in the long run. They now recognize Swetha, are familiar with Equinix, and this familiarity can be invaluable.

In the past, we often found ourselves bogged down by the technicalities of a platform. Concerns ranged from whether the system could handle the assessment load to ensuring the right candidates were being targeted, and troubleshooting any potential glitches. This time, thanks to HackerRank, the experience was seamless. 

From a branding perspective, the convenience cannot be stated enough! I’m not sure about the exact time it took, but updating content on the platform was quick and straightforward. We didn’t need to call in our brand team or liaise with colleagues in Singapore. Essentially, the platform is user-friendly, allowing even someone new from our team to hop on, make necessary edits, and push content without hurdles.

• Have you received any feedback from your candidates and hiring managers about their experience during the process?

Swetha Harohalli Ravikumar: Till now, the overall sentiment has been positive. And one positive sign, from my perspective, is that we received minimal queries during the challenge. In past experiences with similar events, I’ve often been completely bogged down with questions about registration, login issues, and procedural concerns by candidates. This time, there were barely any, except for a few on the challenge day from students who might’ve missed checking their emails. This lack of issues suggests that the platform was user-friendly and seamless for most participants, from registration to the challenge completion.

Dennis Wilfred: The feedback from candidates has also been encouraging. We prioritize getting perspectives from both selected and rejected candidates. One significant benefit of HackerRank is its transparency; candidates gain clear insights into the challenges we aim to address. They appreciate the consistency in our assessment approach, which contrasts with the varied questions they might encounter from different interview panelists.

In particular, the coding exercise – where candidates are given a task and later present their solution – has been transformative. It provides them with a tangible sense of the problems we tackle. I was initially skeptical about its reception, especially among data science candidates who often prefer direct discussions with team leaders. However, the feedback has been outstanding, making the entire HackerRank experience extremely positive for us.

• How’s the quality and fit of candidates improved since you started using HackerRank Engage?

At first, introducing HackerRank faced some resistance. It was a new initiative and represented a shift in our hiring approach. However, once the hiring managers began to observe the high-quality candidates flowing through the pipeline post-assessment, their perspective shifted. The caliber of these candidates was precisely what they were looking for.

We initially piloted HackerRank with a few managers. After witnessing its success, we shared these success stories with other leaders, and it is now widely adopted across departments. Now, my team conducts regular check-ins every quarter with the hiring managers who have used HackerRank to ensure it consistently meets our standards. The feedback, so far, has been overwhelmingly positive.

We’ve actively sought areas of improvement and have received constructive feedback from managers across various sectors, from data science to full-stack engineering. The positive results in India also affected its implementation in Poland and a pilot in the US. One manager, who had experienced HackerRank’s benefits in India and was later hiring in the US, advocated for its use there, and is now leading its adoption in Singapore too.

What stood out for us was the idea of continuous engagement. It wasn’t about a single touchpoint; it was about creating a sustained presence. Through various platforms, like dedicated web pages and consistent email communications, we ensured that Equinix remained top of mind for these students over extended periods.

Our internal talent marketing team has been instrumental in these outreach efforts, amplifying our presence and intent. Given our substantial footprint in the IT sector, especially with a significant number of engineers based in Bangalore, most of our roles naturally cater to this skill set. 

We also made a conscious move towards hosting a diversity hackathon. One of our overarching objectives is to foster a more diverse team at Equinix, especially in our Bangalore branch. These were some of the driving factors behind our initiatives. 

The post Redefining University Hiring: How Equinix Leveraged HackerRank to Attract Top Talent at Scale  appeared first on HackerRank Blog.

In today’s fiercely competitive job market, crafting an exceptional candidate experience is more than just desirable—it’s essential. This holds true even amidst economic uncertainties characterized by layoffs, hiring freezes, and budgetary constraints. In such a climate, excelling in candidate interactions is an imperative. It necessitates that hiring teams refine their approaches, ensuring every candidate interaction is characterized by efficiency, empathy, and a steadfast dedication to providing an exemplary experience.

Sprout Social stands out in its efforts to create people-centric, data-driven candidate experiences. Established in 2010, Sprout Social has emerged as a career destination for technology worldwide. To gain insights into their approach to enhancing candidate experience, we spoke with Miguel Zambrano, Recruitment Enablement Program Manager at Sprout Social.

• What makes for an exceptional candidate experience? What characteristics do you think are vital here?

In today’s competitive job market, I believe that fostering a positive candidate experience begins with clear and transparent communication throughout the interview process. Candidates really crave information, and as recruitment practitioners, we feel it’s our duty to keep them informed and guide them seamlessly through the process. It’s also crucial to recognize that each candidate is unique.

We prioritize personalized interactions because no one wants to feel like just another cog in the machine. By tailoring our approach to understand and address each candidate’s motivators, behaviors, and individual circumstances, we ensure that every person feels seen and heard during their journey with Sprout Social.

• How do you ensure that candidates receive timely and clear communication throughout the hiring process? Can you give us some examples?

In our recruitment process, we take advantage of automated features within our applicant tracking system to streamline communication with candidates efficiently. We employ customizable templates which allow us to tailor our communication and meet each candidate’s unique requirements. While automation helps us to enhance our workflow, we always strive to maintain the human touch that’s so essential in recruitment. Balancing the benefits of technology with the warmth of personal interaction is a priority for us.

These customizable templates are especially useful for our engineering organization. We have developed them based on specific engineering personas. When we find a candidate who aligns with these personas, we ensure that we send them communications designed to capture their interest and effectively convey what Sprout Social offers for that particular engineer.

• What methods do you use to personalize the candidate experiences and make candidates feel valued?

Part of our dedication to delivering an outstanding candidate experience involves our recruitment teams actively engaging with candidates both before and after their final interviews. Before the interviews, we focus on preparation — we address any technical concerns and provide detailed insights into who the candidates will be meeting with. Post-interview, our recruitment partners meet with candidates to gather feedback, answer any lingering questions, and align on the role’s expectations and timelines. This is a crucial moment for candidates to decompress and share their impressions of the interview process.

At Sprout Social, we are deeply committed to Diversity, Equity, and Inclusion (DEI), which is a core value and an integral part of our working culture. During the final interviews, we offer candidates the chance to connect with members from our various community resource groups. This not only gives them a glimpse into our dynamic company culture but also reinforces that Sprout Social is more than just a workplace — it’s a diverse community. We strive to foster meaningful connections and ensure that candidates experience a sense of belonging throughout their journey with us.

• How do you and your teams leverage HackerRank in enhancing candidate experiences? 

HackerRank has proven to be an incredibly intuitive and robust platform for our candidates. It’s equipped with powerful tools that allow for an efficient demonstration of a candidate’s coding skills and professional experience. In our live interviews, we make the most of HackerRank’s features, such as the IDE, virtual whiteboarding, and diagramming tools. The platform’s centralized nature ensures a smooth interview process, and its user-friendly interface is advantageous not only for candidates but also for us as recruitment practitioners and hiring managers, enhancing our ability to effectively assess skills. This makes HackerRank an invaluable asset in our recruitment arsenal.

We’ve also noticed that many candidates are already acquainted with HackerRank, and this recognition undoubtedly adds a layer of comfort to the interview process. It’s reassuring for them to interact with a platform they’ve previously heard of or used. The established brand presence of HackerRank certainly plays a role in easing candidate anxiety during interviews.

• How do you incorporate feedback from candidates to improve your hiring processes? Does the candidate feedback on assessments also help you iterate hiring and experience strategies?

In our pursuit of excellence, we consistently seek feedback from our candidates through surveys disseminated via our ATS. Analyzing this feedback is integral to our practice—it’s pivotal in our mission to deliver a world-class candidate experience. We maintain and rigorously measure our performance against high standards to ensure we’re providing this top-tier experience.

In response to candidate feedback, we’ve developed what we call ‘candidate prep guides’ to ensure thorough preparation for interviews. These guides are customized to both the candidate and the role, detailing each session’s structure, topics, and evaluation criteria. We firmly believe that equipping our candidates to present their best selves during the interview fosters more dynamic discussions and overall satisfaction. Ultimately, our goal is to alleviate interview anxiety and set up each candidate for success, leading to better outcomes for all involved.

Due to our candidate-first approaches we’ve observed a positive trend of candidates returning to us. The open-ended feedback in our surveys often highlights how candidates, even those who were not selected, consider our process the best they’ve experienced. They express enthusiasm about the prospect of reapplying to Sprout Social. It’s been quite revealing to read comments from candidates who, despite not getting the role, still rated the experience very highly. This has been a clear indicator that our approach is resonating well with candidates, marking the effectiveness of our processes.

HackerRank also helps greatly in this regard; we place great importance on reviewing feedback from both candidates and interviewers. This input is invaluable, and we conduct regular audits to ensure we’re making the best use of HackerRank, which fits seamlessly into our workflow thanks to the excellent collaboration with our Customer Success Manager (CSM) at HackerRank. This continuous feedback loop is vital, as it allows our recruitment process to evolve and adapt to the needs of our candidates and the internal teams using the platform.

For instance, sometimes interviewers might encounter a technical issue they need to troubleshoot shortly before an interview. We’ve taken such feedback seriously and ensured that internally, all our interviewers, especially for roles like associate software engineers, have the necessary resources. This includes having access to support contacts, help pages, and a detailed guide on troubleshooting—provided by our CSM at HackerRank—so that nobody feels unprepared or in a scramble right before an interview. This is just one way we’ve turned the feedback we’ve received into proactive steps to improve our process.

• How does Sprout Social create seamless interview and assessment experiences for tech candidates?

Central to our recruitment philosophy is a human-centric approach to hiring. Our priority is to keep candidates well-informed about their progress and set clear expectations at each stage to mitigate the stress typically associated with interviews. Crafting a comfortable and supportive environment is fundamental to our method. 

When it comes to our tech candidates, preparation is crucial. We provide them with detailed interview prep guides ahead of their face-to-face interviews, alongside a specially designed HackerRank new user guide. Our aim is to ensure candidates are well-acquainted with the platform, easing some of the inherent tension of job interviews. This dedication to a smooth experience is reinforced by our tech stack, which includes efficiency-enhancing tools like HackerRank, facilitating a streamlined journey through our recruitment process.

• What steps do you take to minimize bias and promote inclusivity in candidate interactions?

At Sprout, we’re deeply committed to nurturing a culture rich in diversity, equity, and inclusion, and this ethos is embedded right from our onboarding process. Every new Sprout employee is required to complete bias training, reinforcing our dedication to an inclusive workplace. Additionally, our interviewers and hiring managers undergo comprehensive interview training, which includes a focus on recognizing and addressing bias, before they conduct any interviews.

Our applicant tracking system (ATS) is instrumental in minimizing bias throughout the recruitment process. It’s designed with numerous features that act as proactive reminders and guides for DEI best practices during various recruitment stages, such as sourcing candidates, job creation, interviewing, referrals, and extending offers, ensuring we stay true to our values every step of the way.

Moreover, in partnership with our DEI team, we have initiatives that visibly affirm our commitment to DEI, prominently displayed on every user’s ATS portal interface. This reinforces our resolve to weave DEI into the very fabric of our organization. We also publish a DEI report and share it on our company website, offering more insights into our DEI efforts. 

• Could you share some innovative strategies Sprout Social employs to engage candidates, ensuring they have a positive experience regardless of the outcome?

We’ve focused on creating a holistic and engaging recruitment process by weaving our employee value proposition into various stages of recruitment. This includes unique initiatives like our engineering coffee chats, the candidate personas, DEI coffee chats, and proactive outreach from hiring managers. These efforts illustrate our distinctive approach to recruitment.

Our employee value proposition, “see work differently,” is a philosophy that permeates our culture. For example, we’ve established focus days and hours, recognizing the importance of uninterrupted time for deep work. During these periods, we avoid scheduling internal meetings, a practice that has significantly transformed our work dynamics.

For example, at Sprout Social, our EVP is ‘See Work Differently,’ and we’ve embraced creative methods to weave this into our processes. For instance, we focus on communicating both to our candidates and our internal engineering team the significant impact they can make at Sprout. We launched an initiative where we created a blog crafted by engineers for engineers, showcasing the engineering journey at Sprout. It features engaging stories like the progression of an entry-level engineer and insights from different CRG groups, such as women in tech or other underrepresented genders in tech, offering a glimpse into their experiences and growth within Sprout.

Recognizing the growing trend of remote and hybrid work models, Sprout fully embraces these for our distributed teams. We empower our employees to excel in environments where they feel most comfortable. Another aspect of how we “see work differently” is our robust culture of feedback, equipped with tools and training for both giving and receiving feedback. This culture promotes open communication in all directions – top-down, bottom-up, and peer-to-peer.

Specifically addressing our employee value proposition, our recruitment team collaborated with our brand creative and engineering enablement teams to develop a targeted marketing strategy. This strategy is centered around candidate personas – fictional profiles with distinct needs and traits. These personas guide our recruitment team, interviewers, and hiring managers, helping them understand candidates better and effectively communicate the benefits of joining Sprout.

These personas were meticulously developed by our brand creative researchers using key insights and data. We tested them with internal and external participants, gaining valuable insights into candidates’ motivations, preferred tools, and reasons for applying or not applying to a role.

At Sprout, we strive to empower candidates, ensuring they feel they’re evaluating us just as much as we’re assessing them. We aim for a mutually beneficial relationship, fostering a sense of enthusiasm and commitment towards joining our team. This approach significantly contributes to our success in attracting passionate candidates.

• Could you share examples of how AI has successfully improved the candidate experience and recruitment results?

The integration of AI has been a transformative force in recruitment, significantly enhancing and streamlining our processes. Recognizing its potential, we focused on making our recruitment team comfortable with this technology. To demystify AI and its role in recruitment, we conducted workshops and provided comprehensive resources. 

One significant application is using AI by our recruitment partners to generate Boolean strings for niche roles. This approach helps us extend our reach and build stronger candidate pipelines; AI can also help us optimize our messaging to fit within character limits without sacrificing key selling points. Embracing AI strategically has improved our efficiency and our ability to attract top-tier talent. Our goal with AI is to eliminate manual tasks, freeing us to be more creative and strategic where it matters most. This balance between AI efficiency and human creativity is where we’ve found great success.

• What advice would you give to other leaders aiming to improve their candidate experiences?

My primary advice for enhancing candidate experience revolves around the critical importance of active listening. Much like customer data is vital in sales, candidate feedback is crucial in recruitment. By quantifying and understanding what candidates say about our process, we gain valuable insights that allow us to make informed adjustments. In recruitment, every conversation and interaction serves as a significant data point. This information should guide leaders to make decisions that are not only informed but also data-driven.

Recognizing the diversity of candidate preferences, it’s essential to stay attuned to job market trends. Lastly, I believe that building a connection early in the recruitment process is fundamental. We view our company as a career destination, and this starts with aligning with candidates from the beginning. Tailoring our approach to resonate with candidates’ motivators and drivers ensures a more personalized and positive experience throughout their journey.

The post Optimizing Hiring Efficiency: Sprout Social’s Strategic Approach for a World Class Candidate Experience appeared first on HackerRank Blog.

In a few of our recent webinars, we’ve been polling attendees to understand where their sentiments lie regarding AI. And we’ve found some interesting results.

AI for me

Strong majorities embrace using AI to increase their hiring teams’ productivity. In a June webinar, 84% agreed, and 41% strongly agreed.

In a customer webinar hosted in July, 74% agreed.

Overall, we see a clear openness to using AI to help hiring teams be more productive.

What does that mean? In the context of the webinar, it means using AI to take on some more time-consuming tasks, such as analyzing the quality of a candidate’s code or producing a draft interview summary. Viewed through that lens, a little AI help sounds pretty nice. 

Want to hear more about how we see AI powering the next generation of technical interviews? Be sure to check out our on-demand webinar: How HackerRank is Leading AI-Powered Hiring.

But not for thee…

The second question we’ve been asking is whether candidates should be able to use AI tools during coding tests.

Hiring teams using AI to be more productive? Totally cool.

Candidates using AI during coding tests? Hold on just a minute.

In our How HackerRank is Leading AI-Powered Hiring webinar, we found sentiment evenly divided. 39% of attendees agreed candidates should be able to use AI tools, and 43% disagreed, with 19% on the fence.

This tracks with the many conversations we’ve been having over the past several months. Cases can be made both for and against AI use in assessments. If AI can do the work for someone, how can the hiring team be sure the candidate actually has the skills for the job? Isn’t evaluating those skills the entire point of a coding test?

On the other hand, if a hire is going to be working with AI on the job, wouldn’t allowing them to work with AI in the test environment provide a more real-world assessment of their skills?

In a July customer webinar, responses were decidedly more one-sided. Only 19% of attendees favored candidates using AI tools, and 70% opposed. We also noted a steep jump in those strongly opposing candidate AI use: 32% compared to just 14% in the earlier poll.

Embrace AI on your own terms

We don’t expect the question of AI’s role in assessments to be settled any time soon, if ever. What works for one company, or even one role, may not work for another. It’s why we’re designing our AI enhancements to be flexible and customizable, rather than trying to force fit a one size fits all approach.

If you want to learn more about what we’re building, visit HackerRank AI to get the rundown and join the waitlist. And if you want the full story, as told by our AI experts, watch the on-demand webinar, How HackerRank is Leading AI-Powered Hiring.

Where do you stand? Should candidates be able to use AI tools in coding tests? Why or why not?

The post Poll Results: AI Acceptance Varies By Use Case appeared first on HackerRank Blog.

What’s going on?

Since ChatGPT came onto the scene in late 2022, test after test has proven vulnerable to the wiles of generative AI. The initial GPT-3.5 model was impressive enough, and the more advanced GPT-4 has shown an even greater proficiency for test-taking. Name a large, well-known test, and ChatGPT has probably passed it. In addition to bar exams, SATs, and AP exams, ChatGPT has also passed 9 out of 12 AWS certification exams and Google’s L3 engineer coding interview. 

At HackerRank, we’ve seen firsthand how AI can bypass MOSS code similarity, the industry standard for coding plagiarism detection. 

All of these sudden vulnerabilities can seem scary for those administering tests. How can you trust the answers you’re getting? If your tests rely heavily on multiple choice questions, which are uniquely vulnerable to large language models, how can you revise test content to be more AI resistant?

These developments are worrying for test-takers, as well. If you’re taking a test in good faith, how can you be sure you’re getting a fair shake? Interviewing is stressful enough without having to wonder if other candidates are seeking an AI-powered advantage. Developers deserve the peace of mind that they’re getting a fair shot to showcase their skills. 

What’s our stance?

At HackerRank, we’ve done extensive testing to understand how AI can disrupt assessments, and we’ve found that AI’s performance is intrinsically linked with question complexity. It handles simple questions easily and efficiently, finds questions of medium difficulty challenging, and struggles with complex problems. This pattern parallels most candidates’ performance. 

However, creating increasingly intricate questions to outwit AI isn’t a sustainable solution. Sure, it’s appealing at first, but it’s counterproductive for a few reasons. 

• First, this could potentially compromise the core value of online assessments, weakening the quality of talent evaluation. More complex questions don’t automatically translate into better signals into a candidate’s skills. They take longer to answer, which translates into either longer assessments, or fewer questions (and fewer signals to evaluate). 
• Second, it would certainly degrade the candidate experience by focusing on frustrating AI rather than on giving developers a chance to showcase their skills. Losing sight of the developer experience tends to diminish that experience, which could result in more candidates dropping out of the pipeline. 
• Third, it would set up a game of perpetual leapfrog as more advanced AI models solve more complex problems, and even more complex problems are created to trip up more advanced AI. 

Instead, our focus remains on upholding the integrity of the assessment process, and thereby ensuring that every candidate’s skills are evaluated fairly and reliably. 

Introducing our new AI solvability indicator

Upholding integrity means being realistic—and transparent. This means acknowledging that there are assessment questions that AI can solve. And it means alerting you when that is the case, so you can make informed decisions about the content of your assessments. 

That is why we are introducing an AI solvability indicator. 

This indicator operates on a combination of two criteria. 

1. Whether or not a question can be fully solved by AI.
2. Whether or not that solution is picked up by our AI-powered plagiarism detection. 

If a question is not solvable by AI, it does not get flagged. Likewise, if a question is solvable, but the answer triggers our plagiarism detection model, it does not get flagged. The question may be solvable, but plagiarism detection ensures that the integrity of the assessment is protected. 

If a question is solvable by AI and the solution evades plagiarism detection, it will get flagged as AI Solvable: Yes. Generally, these questions are simple enough that the answers don’t generate enough signals for plagiarism detection to be fully effective. 

Questions flagged as AI solvable will be removed from certified assessments, but may still appear in custom assessments, particularly if those assessments have not been updated in some time. 

If you’re browsing through questions, you can also select to hide all AI-solvable questions, just as you can hide all leaked questions. 

What else is HackerRank doing?

Beyond the transparency of the AI solvability indicator, we are building in measures to actively ensure assessment integrity. These include: 

• AI-powered plagiarism detection. Our industry-first, state-of-the-art plagiarism detection system analyzes dozens of signals to detect certain out-of-bounds behavior. With an incredible 93% accuracy rate, our system repeatedly detects ChatGPT-generated solutions, even when they’re typed in by hand, and even when they easily bypass standard detection methods. 
• Certified assessments. Let us handle assessment maintenance. Our certified assessments are out-of-the-box tests curated and maintained by HackerRank experts. We take on all the upkeep, including keeping content current and flagging and replacing any leaked or AI-solvable questions. 
• Expanded question types. We’re expanding question types with formats and structures that are more resistant to AI solutions, such as projects and code repositories. These have the added benefit of being extremely close to the real-world environments and challenges your candidates would face in their daily work, giving you a true-to-life evaluation of their skills. 

What can you do?

No matter where your company stands on AI, we believe it’s best to be transparent about its capabilities. Yes, AI can solve simpler technical assessment questions. We prefer you to know that so that you can take informed actions. 

So what can you do? Every company is coming at AI in their own way, so there’s no one right answer. What works for one organization may not work for another. But broadly speaking, here are some steps you should consider to protect the integrity of your assessments.

• Stay informed. Yes, some technical questions can be solved by AI. At HackerRank, we help ensure assessment integrity through our market leading plagiarism detection and through solvability indicators that give you the transparency you need to deliver fair assessments. 
• Replace solvable questions. When a question in one of your assessments is flagged as AI solvable, a simple course of action is to replace it with an unsolved question from our library. We also recommend looking at the type of question you’re asking, and what you’re hoping to learn from it. It may make sense to replace a solvable question with an entirely different question type.
• Embrace new question types. Newer question formats like projects and code repos are more resistant to AI, and their close resemblance to real-world scenarios gives you a truer-to-life evaluation of how a candidate would perform in their daily work. 
• Take advantage of certified assessments. Don’t want to deal with maintaining and updating assessments? Let us do it for you. With certified assessments, HackerRank experts handle all of the content curation and monitoring, including replacing any leaked or AI solvable questions.
• Leverage HackerRank professional services. Have special needs for your assessments? Engage our experts for monitoring and content creation customized to your specific business objectives. 

Ensure assessment fairness and your own peace of mind

Ensuring assessment integrity in a time of rapidly advancing AI can seem difficult. You can only dial up question complexity so far before it starts to degrade the assessment experience and even compromise the value of assessments in finding qualified talent. That’s why we’re focused on reinforcing key pillars of assessment integrity, including our industry-leading AI-powered plagiarism detection, certified assessments, and solvability indicators that give you the transparency and signals you need to make the best decisions about your assessments. 

Be sure to check out our plagiarism detection page to go into more detail about how HackerRank is ensuring assessment integrity. 

The post AI Can Pass (Some) Test Questions. Now What? appeared first on HackerRank Blog.

In our most recent webinar, How HackerRank is Leading AI-Powered Hiring, Principal Product Manager Ankit Arya and Senior Director of Product Marketing Danielle Bechtel gave customers a first look at new and upcoming products that let companies bring AI into their hiring process—on their own terms.

While there’s no substitute for watching the webinar on demand, here’s a taste of what went down:

3 developments in HackerRank AI

1 – AI-Powered Plagiarism Detection is live

HackerRank’s industry-first AI-powered plagiarism detection system is live and available to all HackerRank customers. By analyzing dozens of unique signals, our new plagiarism detection model detects suspicious activity with far greater reliability and fewer false positives than industry standard methods, like MOSS code similarity. 

2 – AI is about to make hiring teams’ lives easier

Several upcoming platform features promise to make hiring teams’ lives a bit easier. For example, AI will soon be able to review candidate code quality across several metrics such as efficiency and modularity, and provide a rationale for its analysis. AI will also be able to help members of the interview team provide more accurate interview summaries faster, using transcripts to build a first draft that can be refined before submission. 

3 – AI is coming to the assessment experience

HackerRank customers are fairly divided on AI’s role in assessments. Some want—or need—to keep AI at arm’s length. Others want to use it, and want to see how their candidates use it. To allow companies to embrace AI on their own terms, we’re building AI assistance into the assessment experience. Furthermore, the AI assistance will be highly customizable, from limited AI that can onboard a candidate to a codebase, to fully open AI that can engage in pair programming and code generation. 

At the end of the discussion, we held a live Q&A to chat through questions from the audience. Here are five of the top questions we heard—and how we’re thinking about them in response. 

Top 5 questions from hiring teams

The following responses are from the perspective of Ankit Arya, our principal product manager. His answers have been edited for length and clarity. 

1 – Is ChatGPT ready for primetime code complexity?

Base ChatGPT, the GPT-3.5 Turbo model, is not as good for programming. But GPT-4, Bard, and Anthropic’s models are getting to a place where they’re real coding helpers as you’re building software. 

Teams still need human creativity and developers who understand code, but AI can help take care of some of the more tedious tasks. For example, if you wrote a piece of a function and you want it to do error handling, you can have ChatGPT manage that for you. Of course you still need to review it, because you’re ultimately responsible for deploying it in production. But it can be a great assistant and enhance productivity.

2 – Can you talk more about plagiarism detection and 93% reliability? How do you check false positives? How do you even get that information? And has any other third party validated these claims?

The system has been in limited availability and we’ve run thousands of tests to make sure the system is performing at the level that we’re claiming. We’re also looking at feedback  from customers who’ve been using this product, and that feedback’s been really amazing. So that’s really where we are coming from when we define that internal benchmark. 

We’ve also been audited by an external third party, because it does come under the purview of the NYC law. We’ve gone through the audit process, so the system is ready for you to use. 

3 – HackerRank’s plagiarism detection system will get better over time because it’s built on AI. Can you talk more about that?

These systems are built with training data. Imagine when you’re a kid. How do you learn things? Someone shows you an image of an apple and tells you it’s an apple. Teachers give you a lot of examples and a label, and you start building associations, so you can recognize an apple.

This is how AI models learn, as well. Only they’re not as good at it as humans. We just need to see a thing one or two times, and we’ve got it. I could show you any apple, and you’ll identify it with very high accuracy. AI systems need a lot more data. So in this case, they would need a lot more images to make an accurate, reliable prediction. 

When we say the system gets better over time, this is what we mean. The more customers use it, the more feedback they provide, the more training data the system can ingest, further increasing its accuracy.

4 – Lots of people are interested in the interview assistant. What does that look like in the long term? Is this something you see integrating into an ATS?

Yes. Over the long term, we want to get to where the interview assistant does most of the work, and where we’re delivering it to you in your ATS. We don’t want AI making decisions, so imagine this more like AI doing 80-90% of the work for you, compiling the summary that you’d have to spend an hour doing. Now you would be spending 10 minutes reviewing it, making any changes, and then submitting it. 

But we absolutely imagine the system becoming way more integrated into the workflow than it is now, depending on what ATS you’re using.

5 – How does AI in advanced plagiarism handle copy/paste? Are there any plans to disable that functionality altogether?

No, there are no plans to disable copy/paste. I don’t think that’s something we’d ever want to do. To bring a little more clarity, you can’t copy questions. So when you talk about copy/paste, it’s really in the editor window. We provide a proctoring feature that’s essentially copy/paste tracking. And just because someone pasted, doesn’t mean they plagiarized. It’s just one of the signals the model considers. 

For example, someone might be solving a program question, but forgot how to insert a key in a Python dictionary. Simple, basic things just become signals into the model. What we’re really looking for are large patterns of cheating behavior. Is the full solution being pasted in? Or large chunks of code? So whether copy/paste triggers a plagiarism flag depends on the context of how much was copy/pasted and what was copy/pasted. 

Get the full story

These questions only scratch the surface. Be sure to watch the full webinar to take in the full Q&A session and get more context around HackerRank’s new and upcoming AI products.

And if you want to be among the first to gain access to our future AI releases, be sure to sign up for the HackerRank AI waitlist at hackerrank.com/ai.

The post All Things AI: Here’s What You Missed From the HackerRank AI Webinar appeared first on HackerRank Blog.

25% of technical assessments show signs of plagiarism. 

While it’s impossible for companies to fully prevent plagiarism—at least without massively degrading the candidate experience—plagiarism detection is critical to ensuring assessment integrity. It’s important that developers have a fair shot at showcasing their skills, and that hiring teams have confidence in the test results. 

And the standard plagiarism detection method used by, well, everyone, is MOSS code similarity.

MOSS Code Similarity

MOSS (Measure of Software Similarity) is a coding plagiarism detection system developed at Stanford University in the mid-1990s. It operates by analyzing the structural pattern of the code to identify similarity, even when identifiers or comments have been changed, or lines of code rearranged. MOSS is incredibly effective at finding similarities, not just direct matches, and that effectiveness has made it the de facto standard for plagiarism detection. 

That doesn’t mean MOSS is flawless, however. Finding similarity doesn’t necessarily translate to finding plagiarism, and MOSS has a reputation for throwing out false positives, particularly when faced with simpler coding challenges. In our own internal research, we’ve found false positive rates as high as 70%.

AI changes the game

While not perfect, MOSS has been a “good enough” standard for years. Until the advent of generative AI tools like ChatGPT. 

ChatGPT has proven effective at solving easy to medium difficult assessment questions. And with just a bit of prodding, it’s also effective at evading MOSS code similarity. Let’s see it in action:

Step 1: We asked ChatGPT to answer a question and it did so, returning a solution as well as a brief explanation of the rationale. 

Step 2: Next, we directly asked ChatGPT to help escape MOSS code similarity check, and it refused.

Step 3: However, with some creative prompting, ChatGPT will offer unique approaches. And the way that ChatGPT’s transformer-based model works, it generates distinct answers every time, giving it a huge advantage in bypassing code similarity detection. 

Here are three different prompts and three totally different approaches. Note that ChatGPT transforms many variable names from the initial solution to evade code similarity checks.

Step 4: The moment of truth! When we submitted the revised answer through plagiarism detection, it passed cleanly. 

What’s the implication? 

Basically, MOSS code similarity checks can be easily bypassed with ChatGPT. 

Time to panic?

If MOSS code similarity can be bypassed, does that mean that technical assessments can no longer be trusted?

It depends. 

On one hand, it’s easier for candidates to bypass the standard plagiarism check that the entire industry has relied upon. So, yes, there is a risk to assessment integrity.

On the other hand, plagiarism detection has always been a compromise between effectiveness and candidate experience. MOSS is not intrusive, but its high false positive rates render it less definitive than it could be. Ultimately, it’s not really detecting plagiarism. It’s detecting patterns in the code that could be plagiarism.

Move over, MOSS

What happens now?

Plagiarism detection gets rethought for the AI era. Expect companies to scramble for better versions of MOSS, more complex questions, different question types, and more to make up the difference. 

At HackerRank, we’ve taken a different approach. While we’re always improving our question library and assessment experience, we’ve completely rethought plagiarism detection. Rather than relying on any single point of analysis like MOSS Code Similarity, we built an AI model that looks at dozens of signals, including aspects of the candidate’s coding behavior. 

Our advanced new AI-powered plagiarism detection system boasts a massive reduction in false positives, and a 93% accuracy rate. In real-world conditions, our system repeatedly detects ChatGPT-generated solutions, even when those results are typed in manually, and even when they easily pass MOSS Code Similarity. 

What happens when the example shown above gets submitted through our new system? It gets flagged for suspicious activity. 

Clicking into that suspicious activity reveals that our model identified the plagiarism due to coding behaviors.

What’s more, hiring managers can replay the answer keystroke by keystroke to confirm the suspicious activity. 

There’s nothing even close to it on the market, and what’s more, it’s a learning model, which means it will only get more accurate over time.

Want to learn more about plagiarism detection in the AI era, MOSS Code Similarity vulnerability, and how you can ensure assessment integrity? Let’s chat!

The post ChatGPT Easily Fools Traditional Plagiarism Detection appeared first on HackerRank Blog.

Artificial intelligence is no longer just a buzzword, but a powerful field that is changing the tech world in countless ways. From voice assistants and self-driving cars to personalized healthcare and fraud detection, AI is transforming industries — and technical teams — across the board. 

But perhaps no field will be as dramatically impacted by AI as software development itself.

For decades, software development has been a largely manual and labor-intensive process, with developers spending countless hours writing, debugging, and maintaining code. However, with the advent of AI, much of this work can now be automated, freeing up developers to focus on higher-level tasks that require their creativity and expertise. Moreover, AI is being used to enhance various aspects of software development, such as predicting and preventing bugs, generating code, and optimizing performance.

Whether you’re a hiring manager looking to hire the best talent for your AI and software development team, or a developer looking to stay ahead of the curve, this article will provide valuable insights and practical advice for navigating the exciting and rapidly-evolving world of AI and software development.

The Impact of AI on Software Development

AI is already having a significant impact on the software development landscape, and this impact is only expected to grow in the coming years. Some of the ways in which AI is transforming software development include:

• Automating Repetitive Tasks: As mentioned earlier, AI-powered development tools can help automate repetitive and time-consuming tasks, freeing up developers to focus on more complex and creative work. For example, AI can be used to generate code, detect bugs, and optimize performance, among other tasks.
• Improving Efficiency and Accuracy: AI can also improve the efficiency and accuracy of software development. By analyzing large amounts of data and identifying patterns and insights, AI can help developers make better decisions and avoid errors.
• Enhancing User Experience: AI can be used to create more personalized and intuitive user experiences. For example, chatbots powered by AI can help customers get answers to their questions quickly and easily, while recommendation engines can suggest products or services based on users’ preferences and past behavior.
• Enabling New Applications and Use Cases: Finally, AI is opening up new applications and use cases for software development. For example, AI-powered systems can be used for predictive maintenance, fraud detection, and even autonomous vehicles. In addition, tools like ChatGPT, which can assist with building websites and applications, and GitHub’s Copilot, which serves as an AI-pair programmer, are lowering the barrier to entry for people new to coding by providing real-time feedback and suggestions for how to improve their code and work more efficiently.

Overall, AI is transforming the software development landscape in profound ways, enabling developers to work more efficiently and creatively, and creating new opportunities for innovation and growth. However, with these opportunities also come new challenges and considerations.

Challenges and Considerations for Development in an AI-First World

While AI presents many opportunities for software developers, it also raises new challenges and considerations. Here are some of the key issues that developers must address in an AI-first world:

• Ethical and Legal Concerns: In an AI-first world, development teams will need to be able to identify and mitigate ethical concerns related to AI. Already U.S. regulators are examining ways to tackle the potential harm caused by AI, and these regulations will likely have a major impact on the way this technology is used and developed. AI software systems will need to be transparent, explainable, and fair, and comply with all relevant regulations and standards.
• Bias and Discrimination: AI-powered systems can perpetuate or even amplify biases, leading to discrimination against certain groups of people. Development teams will need to be vigilant in identifying and mitigating these biases, and ensure that their systems are inclusive and accessible to all users.
• Data Privacy and Security: AI relies on large quantities of data, and developers must ensure that this data is collected, stored, and used in a secure and responsible manner. They also have to protect users’ privacy and sensitive information, and comply with data protection regulations.
• Skills and Training: Developing and implementing AI-powered systems requires a different skill set than traditional software development. Development teams will need to hire or retrain for skills in data science, machine learning, and AI-powered development tools.
• Scalability and Maintenance: Finally, AI-powered systems can be complex and difficult to maintain, particularly as they scale up and handle larger amounts of data and users. Developers must design their systems with scalability and maintenance in mind, and implement strategies to ensure that they remain reliable and efficient over time.
• Collaboration With Data and AI Teams: In an AI-first world, developers will need to work more closely with data scientists and machine learning engineers to develop and implement AI-powered applications. This requires a good understanding of the principles and tools of data science and machine learning, as well as the ability to work with big data and advanced analytics.

By addressing these challenges and considerations, development teams can ensure that their AI-powered systems are effective, ethical, and secure, and can drive real value for users and organizations. However, doing so will require a collaborative and multidisciplinary approach, with input from machine learning engineers, data scientists, legal experts, and other stakeholders.

AI-First Companies and the Implications for Developers

In recent years, we have seen the emergence of a new type of company – the AI-first company. These are organizations that prioritize the use of AI and data-driven insights to power their business models, operations, and strategies. Examples of AI-first companies include Google, Amazon, Facebook, and Microsoft.

AI-first companies operate differently from traditional companies in several key ways. First, they are more data-driven, using AI to analyze vast amounts of data and generate insights that drive business decisions. Second, they are more agile and adaptable, using AI to quickly respond to changing market conditions and customer needs. They are also more customer-centric, using AI to personalize and optimize the customer experience.

For developers, this shift towards AI-first companies presents both new opportunities and challenges. On the one hand, AI-first companies require developers with strong skills in machine learning, data analytics, and software engineering to build and maintain their AI-powered systems. These companies also value developers who are creative, innovative, and able to work collaboratively across teams and domains.

On the other hand, AI-first companies can also present challenges for development teams. For example, they may require developers to work with large and complex datasets, which can be daunting for those without a strong background in data management. Additionally, they may require developers to work within a fast-paced and constantly evolving environment, where new tools and technologies are being developed and deployed on a regular basis.

To succeed in an AI-first company, developers need to be adaptable, open-minded, and willing to learn and experiment with new tools and technologies. Some of the technologies that are most valuable to AI-first companies include:

• Deep learning frameworks like TensorFlow and PyTorch
• Natural language processing libraries like NLTK and SpaCy
• Big data processing tools like Apache Spark and Hadoop
• Cloud computing platforms like Amazon Web Services and Microsoft Azure
• Software development best practices like agile development, DevOps, and continuous integration/continuous deployment (CI/CD)

Key Takeaways

The rise of AI-first companies is transforming the software development landscape, creating new opportunities and challenges for developers. By staying abreast of the latest trends and developments in this space, developers can position themselves to thrive in this new and exciting era of software development.

At HackerRank, we understand the importance of developing and nurturing the skills of today’s developers, especially in the emerging field of AI and machine learning. That’s why we created a roles directory that helps hiring managers and tech professionals explore various job families and tech roles. Want to learn more about the real-world skills driving the future’s innovation? Visit our roles directory today.

The post How AI Is Shaping the Future of Software Development appeared first on HackerRank Blog.

Are you ready for the AI revolution in coding and software development? Our new Developer Skills: AI report reveals key insights that every developer and tech hiring team should know.

The AI revolution is changing the very nature of what it means to be a developer. Our survey of more than 42,000 global developers in February and March 2023 showed that 82% of developers believe AI will redefine the future of coding. Furthermore, 75% are already adjusting their skills to keep up with this game-changing shift. With such a significant impact on the industry, it’s crucial for both developers and companies to understand these changes and adapt accordingly.

“We’ve entered an AI revolution that is poised to change the very nature of what it means to be a developer and write code,” said Vivek Ravisankar, co-founder and CEO at HackerRank. “I see the result of this revolution as faster innovation than ever before, the democratization of development, and expanded opportunities for developer creativity. And this is just the tip of the iceberg.”

AI Is Already Being Used to Augment Coding Tasks

The report found that developers and employers alike are racing to embrace artificial intelligence in the workplace. Access to AI assistants will transform key elements of development work—automating many repetitive or tedious tasks and creating space for more abstract thinking and creative problem-solving.

“AI is set to become a key part of developer workflows, with the rise of AI assistants like GitHub Copilot and all-purpose tools such as ChatGPT,” said Ankit Arya, Principal Product Manager, AI at HackerRank. “Personally, I use ChatGPT for retrieving information or code snippets while coding, and I find it way more efficient than traditional search engines. AI’s potential lies in augmenting developers’ skills rather than replacing them.”

On the Hiring Front, an Uptick in Demand for AI Skill Sets

Employers, too, must prepare for this AI revolution. They face pressure to find, hire, and nurture teams with the technical skills required to capitalize on new innovation and business opportunities driven by AI advancements. We have seen an 81% increase in the creation of new assessments with AI-related questions on our platform since ChatGPT’s public launch in November 2022, signaling a growing interest in hiring for AI-centered skill sets.

Our report also revealed a gap between the AI skills companies need and the skills they’re currently testing for. Our analysis of nearly 1,000 job descriptions revealed that the most in-demand skills for AI-related roles are machine learning, Python, PyTorch, TensorFlow, deep learning, and AWS. However, companies continue to test for more general and conceptual topics, like problem solving and statistics.

To remain competitive, developers need to adapt their skills, and companies need to refine their hiring practices. With the AI revolution already underway, it’s more important than ever to stay informed and embrace the changes it brings.

Don’t miss the chance to stay ahead of the curve—download the Developer Skills: AI report now and get the insights you need to navigate the AI-driven future of coding and software development.

The post AI Is Changing How Developers Work — and How Companies Hire Skills appeared first on HackerRank Blog.

When ChatGPT launched on November 30, it captured attention in a way that other AI products have never quite managed. 

Within five days, it surpassed one million users (a feat that took Netflix three and a half years), and almost immediately, the speculation began. In between the excitement of experimentation, people expressed concern about what this tool meant for the future of work and, specifically, the future of jobs. Will AI make all white-collar jobs obsolete? Or will it just sweep away writers? Or designers, too? What about lawyers? Software developers?

AI isn’t going to replace developers, but it is going to change software development forever. AI pair programming assistants will become integral to the development process, taking over certain time-consuming tasks, delivering massive productivity gains, and allowing developers to focus on the more creative aspects of their projects. 

AI’s impact on the future of work will be profound, but it will fit within a progression of existing trends. Like past technologies from the PC to the cloud, AI will fuel a surge in demand and experimentation before ultimately becoming a normalized part of the development process. 

As people, we don’t handle change well

The latest uproar over AI is a reminder that humans aren’t good at dealing with the uncertainty of change. We tend to dismiss or catastrophize new technologies. Plato once warned that writing would eliminate the need for memory, which, when you think about it, isn’t that far removed from the tweets and TikToks forecasting that AI will wipe out entire professions.

When considering technology’s impact on jobs, we can draw two main takeaways from past innovation cycles.

First, new technologies tend to create more jobs than they erase. For example, the rise of the PC and the internet wiped out about 3.5 million jobs in the US from 1980 through 2018, according to McKinsey estimates. But that same rise also created 19.3 million jobs, for a net gain of 15.8 million jobs.

Second, new technologies are adopted over a longer period of time than we tend to assume. Technology doesn’t just spring into existence. Jobs don’t just disappear. Major technological shifts can take years to play out. The smartphone, arguably the fastest-adopted technology in human history, took four years to reach 40% market penetration. 

As the U.S. Bureau of Labor Statistics observes, “[the] immediate effects [of technological change] are probably smaller than anticipated and their full impact unfolds gradually over a longer timeframe than recognized”.

It’s critical to note that these are precedents, and there’s no guarantee that what’s happened in the past will hold in the future. AI will probably follow the usual course of innovation. It will probably upend whole sectors and create new ones, eliminate some jobs but create more. And it will probably do so on a time scale that allows us to adapt. 

Just remember that probably does not equal certainly. AI may advance a lot faster than we anticipate, or stall out and have nowhere near the impact most think. Regulated industries and the likelihood of future regulations could impact adoption. There’s also the possibility that AI spins out of control and we end up in some kind of doomsday scenario. 

AI is the next step in the march of progress

ChatGPT didn’t come out of nowhere. AI has been under development for quite some time. One early breakthrough in neural networks, training an algorithm to detect cats in YouTube videos, occurred in 2012. And the Transformer network architecture that underpins today’s large language models (LLMs) was first proposed by Google in 2017. 

Transformer-based LLMs like GPT-3 and DALL-E are writing essays, turning natural language prompts into images and code snippets, and even identifying protein relationships to speed up drug discovery. 

But they’re also building on a steady advance of innovation. 

Programming itself has been evolving into something increasingly resembling English. Python, one of the most popular programming languages today, was once regarded to be as close to written English as they come. ChatGPT moves even closer to English by allowing users to generate code from natural language prompts.

What’s more: technology is always evolving in ways that allow developers to work at increasingly higher levels of abstraction. The higher the level of abstraction, the fewer granular, in-the-weeds details the developer needs to think about. 

Every programming language is an abstraction of the 0s and 1s that all computers actually run on. Cloud is an abstraction, allowing developers to have selective ignorance of the distributed systems they’re deploying to. Containerization, elastic load balancing, low code environments, intelligent auto-complete tools like GitHub Copilot—all of them take some tedious, manual element of work and abstract it away. 

AI represents a next step in these two converging trends; the blurring of programming and natural language, and the progression to higher levels of abstraction.

Where does this take us?

What does the growth of AI mean for the future of work? For developers, it’s going to mean evolving their skills, embracing new technologies, and shifting their conception of what it means to write code. 

AI will abstract away an increasing share of basic but time-consuming coding tasks—think debugging, compatibility testing, and documentation. This coding grunt work can eat up a lot of time, and AI tools like GitHub Copilot, Replit Ghostwriter, Mintlify, and others are already demonstrating significant productivity improvements.

Conversational interfaces like ChatGPT are showing a lot of potential and future versions could become mainstream tools in the development process, particularly for initial prototyping. It’s not hard to imagine such tools shortening development timelines by weeks. In such a scenario, the emerging skill of prompt engineering would become essential for many developers.  

AI advancements will also have a major impact on future developers. Abstraction tends to increase accessibility, and therefore adoption. Think about what graphical user interfaces did for PCs, or what the cloud has done for deploying software. In the field of machine learning, abstractions created by frameworks such as PyTorch and TensorFlow opened up opportunities to students in undergraduate and masters programs, not just PhDs. AI can do the same across the board. It can help new developers get those early wins that can get them hooked. It can enable personalized learning at scale, and it can help established developers extend their skills and learn new ones. 

AI seems poised to fuel a surge in software development. While it remains to be seen how fast AI will be adopted and how far its capabilities will grow, it will bring developers along with it. AI tools will make the field more accessible, deliver productivity gains, and enable developers to focus on more creative and challenging problems. All of that may mean that certain skills, even some we consider foundational, lapse in the coming years. And that is totally natural. That is what innovation is supposed to look like. It’s supposed to take us forward—to empower us to work at higher levels of abstraction.

The post AI Will Change Technical Skills Forever appeared first on HackerRank Blog.

Since its public debut in November, ChatGPT has taken the world by storm. In only five days, it surged to one million users. In just over a month, the valuation of the company behind it, OpenAI, grew to $29 billion.

Across sectors, there’s a growing chorus of questions about the implications of large language models (LLMs) like ChatGPT. Will these AI-enabled tools change education and make essay writing obsolete? Can they generate creative enough ideas to power mainstream ad campaigns? Will tools like ChatGPT provide a viable alternative to traditional search engines?

We’re asking some equally big questions ourselves: How well can ChatGPT actually code? And what impact will LLMs have on the broader world of computer programming? 

AI-powered innovation like ChatGPT is poised to fundamentally change the relationship between developers and coding, including how employers assess technical skills and hire developers. With that in mind, we dove deep into the details of ChatGPT, its impact on skill assessments, and what its development means for the future of technical hiring.

Key Takeaways:

• The coding potential of LLMs has reinforced the need for strategies and tools for upholding the integrity of coding assessments.
• Strong proctoring tools and plagiarism detection systems have become essential, and can help protect even solvable questions. 
• Employers should avoid multiple choice questions and problems that have answers so short that a plagiarism detection system can’t detect when a candidate has received help from a tool like ChatGPT.
• Continued growth of artificial intelligence will redefine the real-world application of coding skills and, in the process, change technical hiring as we know it.
• HackerRank is embracing AI and will pursue innovative ideas that imagine a future of programming in an AI-driven world.

What is ChatGPT?

On a basic level, ChatGPT is an example of a large language model. A large language model is a computer system trained on huge data sets and built with a high number of parameters. This extends the system’s text capabilities beyond traditional AI and enables it to respond to prompts with minimal or no training data.

The goal of ChatGPT’s developer, OpenAI, was to create a machine learning system which can carry a natural conversation. In practice, ChatGPT functions like a search engine or content creation system, synthesizing billions of data points into custom responses. 

Developing a Smart Conversational Agent

The development of ChatGPT incorporated two innovative approaches: 

1. ChatGPT is powered by the well-known ML model GPT3.5. The model is trained to complete the next few words of an incomplete sentence. The main idea behind this model is that, after training against billions of data points, the model starts to understand enough about the human world to complete sentences.
2. ChatGPT uses a human-in-the-loop system to continuously improve and answer questions in a more human-like fashion. OpenAI hired thousands of contractors to write human-like responses to challenging prompts as a way to continuously improve the model. Training the model to answer difficult questions improved ChatGPT’s responses at a remarkable rate.

Now that the training process is complete, users can run ChatGPT on accessible devices. This trait makes it superior to other models like AlphaCode, which are thought to be prohibitively expensive to run even after training is complete.

What Are the Strengths of ChatGPT?

Using the process above, OpenAI trained ChatGPT on almost all human knowledge. This enables ChatGPT to:

• Create never before seen sentences and code. Because it’s seen billions of sentences and lines of code, ChatGPT can synthesize the information it has seen and form answers to questions that can be perceived as novel. However, there’s no guarantee that this code will be correct or optimal.

• Combine ideas that it has seen separately but never in combination. For example, ChatGPT can write an answer to a coding question in the writing style of a specific author. 
• Exhibit a breadth of information. ChatGPT is trained on so much data that it has seen examples of most common situations and their potential variations. This enables it to give specific answers to niche questions or generalized answers based on more specific data.

What Are the Limitations of ChatGPT?

While ChatGPT outputs human-like sentences, and it’s easy to mistake its output as being powered by true intelligence, ChatGPT does have shortcomings. 

In describing the tool’s limitations, OpenAI explained that ChatGPT may occasionally “generate incorrect information” or “produce harmful instructions or biased content.” Industry publications have described ChatGPT as confidently wrong, exhibiting a tone of confidence in its answers, regardless of whether those answers are accurate. 

ChatGPT lacks the ability to fact-check itself or conduct logical reasoning. It often incorrectly answers questions and can be tricked relatively easily. Technologists have also noted its propensity to “hallucinate,” a term used to describe when an AI gives a confident response that is not justified by training data.

How ChatGPT Impacts Assessment Content

As a coding tool, ChatGPT excels at certain types of technical problems—but also has its limitations. A strong content strategy will be necessary to test your current coding challenges and prioritize the questions, and question types, that are less susceptible to AI coding support. 

ChatGPT has probably seen almost all known algorithms. But ChatGPT isn’t just able to answer these algorithm questions correctly. It’s also able to write new implementations of those algorithms, answer freeform questions, and explain its work.

As a result, ChatGPT can answer the following question types with reasonable accuracy:

• Well-known algorithms: It’s safe to assume that ChatGPT has seen and is able to answer all publicly available coding problems on platforms such as LeetCode and StackOverflow. If the algorithm appears in online forums or practice websites, ChatGPT will likely answer it correctly.

• Minor variations of problems. ChatGPT does well on variations that tend to add to the solution rather than change it in any substantial way. The system can, for example, easily reverse the order of an array of numbers.
• Multiple choice questions. When presented with a question and multiple potential answers, ChatGPT can usually identify the correct answer.

For hiring teams who administer coding challenges, that doesn’t mean you should necessarily avoid all questions that ChatGPT can solve. With the right protections in place, even questions solvable by AI can still be reliable. The key is to avoid questions that have answers so short that a plagiarism detection system can’t detect when a candidate has used a tool like ChatGPT. Even so, we are evolving our library with new types of content specifically designed with AI code assistance tools in mind.

Taking all of this into account, there are some actions you can take today to limit your hiring content’s exposure to the risk of plagiarism, including: 

• Avoid easily solved multiple choice questions
• Avoid simple prompts to solve for common or widely available algorithm variants
• Remove questions that require only a few lines of code to solve
• Use proctoring tools and plagiarism detection systems
• Combine coding tests with virtual interviewing tools to add empirical data to the hiring process

Ensuring Assessment and Hiring Integrity

In a world where humans and machines alike can write code, the ability to detect the use of AI-coding tools is invaluable. As such, employers increasingly turn to strategies and technologies that enable them to uphold the integrity of their technical assessments.

Assessment integrity has two core pillars: proctoring tools and plagiarism detection.

Proctoring Tools

One important component of ensuring assessment integrity is to build systems that provide the right proctoring capabilities. 

Proctoring is the process of capturing behavioral signals from a coding test, and its purpose is twofold. First, proctoring tools record data points that support plagiarism detection. Second, proctoring tools also act as a deterrent against plagiarism, as candidates who know that proctoring is in place are less likely to engage in such activity.

The key behavioral signals that proctoring tools often record include:

• Tab proctoring. Monitors if the candidate switches between tabs.
• Copy-paste tracking. Tracks if a candidate pastes copied code in the assessment.
• Image proctoring. Captures and records periodic snapshots of the candidate.
• Image Analysis. Analyzes webcam photos for suspicious activity.

Plagiarism Detection

In addition to proctoring tools, the integrity of an assessment also relies on plagiarism detection. In other words, the ability to flag when a candidate likely received outside help. 

The current industry standard for plagiarism detection relies heavily on MOSS code similarity. Not only can this approach often lead to higher false positives rates, but it also unreliably detects plagiarism originating from conversational agents like ChatGPT. That’s because ChatGPT can produce somewhat original code, which can circumvent similarity tests.

While the launch of ChatGPT caught many by surprise, the rise of LLMs has been a popular topic in technical communities for some time. Anticipating the need for new tools to ensure assessment integrity, HackerRank developed a state-of-the-art plagiarism detection system that combines proctoring signals and code analysis.

Using machine learning to characterize certain coding patterns, our algorithm checks for plagiarism based on a number of signals. Our model also uses self-learning to analyze past data points and continuously improve its confidence levels.

The result is a brand new ML-based detection system that is three times more accurate at detecting plagiarism than traditional code similarity approaches—and can detect the use of external tools such as ChatGPT.

Embracing Artificial Intelligence

As exciting as the launch of ChatGPT has been, LLMs with its capabilities are only the beginning. While it’s hard to predict the future, one thing is certain: AI technology is in a nascent state and will continue to grow at a rapid rate.

In the short term, the key to evolving your hiring strategy hinges on a renewed focus on content innovation and assessment integrity. By combining a strong question strategy with advanced proctoring and plagiarism detection, hiring teams can protect their assessment integrity and hire great candidates.

In the long term, we anticipate that artificial intelligence will redefine developer skills and, in the process, change technical hiring as we know it. 

At HackerRank, our mission is to accelerate the world’s innovation. As such, we welcome this new wave of technological transformation and will pursue innovative ideas that imagine a future of programming in an AI-driven world. 

Frequently Asked Questions

Can Your Plagiarism Detection System Detect Code From ChatGPT?

Yes. Our AI-enabled plagiarism detection system feeds several proctoring and user-generated signals into an advanced machine-learning algorithm to flag suspicious behavior during an assessment. By understanding code iterations made by the candidate, the model can detect if they had external help, including from ChatGPT.

When Will the Plagiarism Detection System Be Available?

The new plagiarism system is currently in limited availability, with plans for general availability in early 2023. If you would like to participate in our limited availability release, please let your HackerRank customer success manager know and we would be happy to enable you.

Can You Validate if My Coding Questions Are Easily Solved by ChatGPT and Provide Replacement Options?

If you would like assistance in verifying how ChatGPT responds to your custom coding questions, we can run a report and provide content recommendations based on the results. Please contact our HackerRank Support Team, who would be happy to help. 

Should I Avoid All Questions That ChatGPT Can Solve? 

No. HackerRank’s proctoring tools and plagiarism detection system can protect even solvable questions. Instead, avoid multiple choice questions and problems with very easy or short answers.

I Still Have Questions About ChatGPT. Who Should I Contact?

If you’re a customer looking for support on plagiarism and its impact on your business, you can contact your customer success manager or our team at support@hackerrank.com.

The post What Is ChatGPT? And What Does It Mean for Technical Hiring? appeared first on HackerRank Blog.